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Abstract 


The  objectives  of  this  work  were  (1)  to  survey  the  marketplace  for  available  tools  that  may  provide 
collaborative  environments  to  support  Synthetic  Environment  based  exercises  and  experiments,  and 
evaluate  the  most  relevant  candidates  and  (2)  to  develop  an  evaluation  method  for  assessing 
collaborative  planning  and  engineering  tools.  A  literature  review  was  conducted,  followed  by 
Subject  Matter  Experts  (SMEs)  interviews.  A  total  of  215  collaborative  tools  were  uncovered. 

In  order  to  develop  an  evaluation  method  for  these  tools,  it  was  realized  that  users  of  collaborative 
planning  and  engineering  tools  would  have  specific  uses  for  the  tools  or  goals  for  the  tools  known 
prior  to  tool  selection.  Further,  specific  requirements  would  be  desired  of  the  tools.  This  led  to  the 
creation  of  the  Evaluation  Matrix  which  was  used  to  evaluate  select  collaborative  tools.  Research 
and  development  opportunities  can  also  be  identified  through  the  evaluation  matrix.  Finally,  a 
number  of  changes  are  proposed  for  the  evaluation  matrix.  The  authors  feel  that  this  project  has 
provided  an  important  first  step  toward  the  technological  support  of  distributed  planning  and 
engineering  teams. 


Resume 

Les  objectifs  du  present  ouvrage  etaient  (1)  d’etudier  le  marche  pour  trouver  les  outils  offerts  qui 
peuvent  procurer  des  environnements  de  collaboration  pour  appuyer  les  exercices  et  les 
experiences  dans  des  environnements  synthetiques  et  d’evaluer  les  candidats  les  mieux  qualifies;  et 
(2)  d’elaborer  une  methode  pour  evaluer  les  outils  de  planification  et  d’ingenierie  en  collaboration. 
Une  analyse  documentaire  a  ete  effectuee,  suivie  d’entrevues  avec  des  experts  en  la  matiere  (EM). 
On  a  decouvert  au  total  215  outils  de  collaboration. 

Afm  d’elaborer  une  methode  devaluation  pour  ces  outils,  on  a  realise  que  les  utilisateurs  des  outils 
de  planification  et  d’ingenierie  en  collaboration  les  utiliseraient  a  des  fins  precises  ou  connaitraient 
les  objectifs  a  atteindre  avant  la  selection  des  outils.  De  plus,  il  serait  souhaitable  d’avoir  des 
exigences  precises  pour  les  outils.  On  a  done  cree  une  matrice  devaluation,  que  Ton  a  utilisee  pour 
evaluer  des  outils  de  collaboration  choisis.  Les  possibilites  de  recherche  et  developpement  ont 
egalement  ete  cemees  grace  a  la  matrice  devaluation.  Enfin,  un  certain  nombre  de  changements 
sont  proposes  pour  la  matrice  devaluation.  Les  auteurs  croient  que  ce  projet  a  constitue  une 
importante  premiere  etape  vers  le  soutien  technologique  des  equipes  de  planification  et  d’ingenierie 
reparties. 


DRDC  Ottawa  CR  2007-206 


I 


This  page  intentionally  left  blank. 


DRDC  Ottawa  CR  2007-206 


Executive  summary 


The  DRDC  Ottawa  Future  Forces  Synthetic  Environments  (FFSE)  section  has  been  established  to 
provide  a  Research  and  Development  (R&D)  centre  of  excellence  in  the  area  of  Synthetic 
Environments  (SE)  and  Capability  Engineering  (CE).  The  overall  objective  is  to  study  and  develop 
requirements  of  a  collaborative  environment  that  would  be  exploited  in  the  development  of  SE- 
based  exercises  and  events  to  facilitate  collaboration  and  data  sharing  during  SE  development. 

The  objectives  of  this  work  are  two-fold: 

•  Survey  the  marketplace  for  available  tools  that  may  provide  collaborative  environments  to 
support  SE-based  exercises  and  experiments,  and  evaluate  the  most  relevant  candidates; 

•  Develop  an  evaluation  method  for  assessing  collaborative  planning  and  engineering  tools. 

The  first  step  in  the  project  was  to  conduct  a  literature  review.  Immediately  following  the  literature 
review,  Subject  Matter  Experts  (SMEs)  were  interviewed  for  the  project.  A  total  of  215 
collaborative  tools  were  uncovered  during  these  stages  of  the  project. 

In  order  to  develop  an  evaluation  method  for  these  tools,  it  was  realized  that  users  of  collaborative 
planning  and  engineering  tools  would  have  specific  uses  for  the  tools  or  goals  for  the  tools  known 
prior  to  tool  selection  (e.g.  Communication  and/or  Design).  Further,  specific  requirements  would 
be  desired  of  the  tools  (e.g.  Usable)  where  only  a  subset  of  requirements  would  be  applicable  to 
each  use/goal.  The  team  attempted  to  maximise  the  orthogonality  between  the  uses/goals  and  the 
requirements.  This  led  to  the  creation  of  the  Evaluation  Matrix  with  the  ‘Uses/Goals’  down  the  left 
(vertical  axis)  and  the  ‘Requirements’  along  the  top  (horizontal  axis). 

The  evaluation  matrix  was  used  to  evaluate  five  collaborative  tools:  SharePoint,  NetMeeting, 
LiveLink,  Groove  and  Google  Docs.  Based  on  these  scores,  SharePoint  ranked  as  the  most 
appropriate  collaborative  tool  overall.  However,  based  on  specific  uses/goals  desired  of  a 
collaborative  tool  another  tool  may  be  better  suited.  Once  a  number  of  tools  have  been  evaluated, 
it  will  then  be  possible  to  select  specific  tools  for  specific  uses/goals,  based  on  their  score. 

Research  and  development  opportunities  can  be  identified  through  the  evaluation  matrix  also  by: 

(1)  looking  at  requirement  scores  and  (2)  looking  at  use/goal  scores.  For  example, the  provision  of 
a  searchable,  writable  database  with  reusable  classes  is  a  poorly  met  requirement.  Examples  of 
poorly  achieved  uses/goals  were  ‘simulation/demonstration’  and  ‘network  development’. 

A  number  of  changes  are  proposed  for  the  evaluation  matrix  developed  in  order  to  better  able  to 
accommodate  comments  about  specific  judgements.  Further  changes  are  suggested  concerning  the 
scoring  system  to  permit  direct  comparisons  between  different  scores. 

The  authors  feel  that  this  project  has  provided  an  important  first  step  toward  the  technological 
support  of  distributed  planning  and  engineering  teams. 
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Sommaire 


La  section  des  Environnements  synthetiques  des  forces  de  l’avenir  (ESFA)  de  RDDC  Ottawa  a  ete 
etablie  pour  foumir  un  centre  d’excellence  en  recherche  et  developpement  (R  &  D)  dans  les 
domaines  des  environnements  synthetiques  et  de  l’ingenierie  des  capacites  (IC).  L’objectif  global 
consiste  a  etudier  et  a  formuler  des  exigences  pour  un  environnement  de  collaboration  qui  serait 
exploite  dans  la  mise  au  point  d’exercices  et  d’activites  fondes  sur  EES  afin  de  faciliter  la 
collaboration  et  le  partage  de  donnees  pendant  la  creation  de  l’ES. 

Les  objectifs  de  present  ouvrage  comportent  deux  volets  : 

•  Etudier  le  marche  pour  trouver  des  outils  existants  qui  pourraient  procurer  des 
environnements  de  collaboration  pour  appuyer  les  exercices  et  les  experiences  dans  des 
environnements  synthetiques  et  d’evaluer  les  candidats  les  mieux  qualifies; 

•  Elaborer  une  methode  pour  evaluer  les  outils  de  planification  et  d’ingenierie  en  collaboration. 

La  premiere  etape  du  projet  etait  d’effectuer  une  analyse  documentaire.  Immediatement  apres  celle- 
ci,  on  a  interviewe  des  experts  en  la  matiere  pour  le  projet.  Pendant  ces  etapes  du  projet,  on  a 
decouvert  au  total  215  outils  de  collaboration. 

Afin  d’elaborer  une  methode  devaluation  pour  ces  outils,  on  a  realise  que  les  utilisateurs  des  outils 
de  planification  commune  et  d’ingenierie  les  utiliseraient  a  des  fins  precises  ou  connaitraient  les 
objectifs  a  atteindre  avant  la  selection  des  outils  (p.  ex.  Communication  et/ou  conception).  De  plus, 
il  serait  souhaitable  d’ avoir  des  exigences  precises  pour  les  outils  (p.  ex.  utilisable)  ou  seulement  un 
sous-ensemble  d’exigences  s’appliquerait  a  chaque  utilisation/objectif.  L’equipe  a  essaye  de 
maximiser  l’orthogonalite  entre  les  utilisations/objectifs  et  les  exigences,  ce  qui  a  abouti  a  la 
creation  de  la  matrice  devaluation  qui  comprend  les  «  utilisations/objectifs  »  a  gauche  (axe 
vertical)  et  les  «  exigences  »  dans  le  haut  (axe  horizontal). 

La  matrice  devaluation  a  ete  utilisee  pour  evaluer  cinq  outils  de  collaboration  :  SharePoint, 
NetMeeting,  LiveLink,  Groove  et  Google  Docs.  D’apres  les  resultats,  SharePoint  est,  dans 
l’ensemble,  l’outil  de  collaboration  le  plus  approprie.  Cependant,  selon  les  utilisations/objectifs 
precis  souhaites,  il  se  peut  qu’un  autre  outil  soit  plus  approprie.  Des  qu’un  certain  nombre  d’outils 
aura  ete  evalue,  il  sera  possible  de  choisir  des  outils  precis  pour  des  utilisations/objectifs  precis  en 
fonction  de  leur  resultat. 

Nous  pouvons  cemer  les  possibilites  en  recherche  et  developpement  a  l’aide  de  la  matrice 
d’ evaluation  en  regardant  les  resultats  relativement  aux  exigences  et  en  regardant  ceux  pour  les 
utilisations/objectifs.  Par  exemple,  la  mise  en  place  d’une  base  de  donnees  interrogeable  et 
inscriptible  avec  des  classes  reutilisables  est  une  exigence  non  satisfaite.  Les  categories  de 
«  simulation/demonstration  »  et  de  «  developpement  de  reseau  »  sont  des  exemples 
d’utilisations/objectifs  non  satisfaits. 

LJn  certain  nombre  de  changements  sont  proposes  pour  la  matrice  devaluation  qui  a  ete  creee  pour 
mieux  tenir  compte  des  commentaires  faits  apropos  de  jugements  particuliers.  D’autres 
changements  sont  proposes  en  ce  qui  conceme  le  systeme  de  pointage  afin  de  permettre  des 
comparaisons  directes  entre  les  differents  resultats. 

Les  auteurs  croient  que  ce  projet  a  constitue  une  importante  premiere  etape  vers  le  soutien 
technologique  des  equipes  de  planification  et  d’ingenierie  reparties. 

Lamoureux,  T.L  et  Rehak,  L.A.  2007.  Review  of  Collaborative  Tools  for  Planning  and 
Engineering.  RDDC  Ottawa  CR  2007-206.  R  &  D  pour  la  defense  Canada  -  Ottawa. 
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1  Introduction 


The  DRDC  Ottawa  Future  Forces  Synthetic  Environments  (FFSE)  section  has  been  established  to 
provide  a  Research  and  Development  (R&D)  centre  of  excellence  in  the  area  of  Synthetic 
Environments  (SE)  and  Capability  Engineering  (CE).  In  their  fullest  application,  these  fields  are 
broad,  wide  reaching  and  interact  with  a  significant  number  of  activities  conducted  by  many 
R&D  groups  in  DRDC  and  at  other  organizations.  Through  this  interaction,  a  number  of  ‘gaps’ 
emerge  that  demand  additional  exploration  and  development;  one  such  area  of  increasing  interest 
is  the  role  of  collaborative  environments  in  supporting  SE-based  exercises  and  experiments. 

Discussions  with  potential  section  clients  have  lead  to  a  survey  of  current  collaborative  planning 
and  engineering  tools  given  the  importance  of  Collaborative  Engineering.  The  overall  objective  is 
to  study  and  develop  requirements  of  a  collaborative  environment  that  would  be  exploited  in  the 
development  of  SE-based  exercises  and  events  to  facilitate  collaboration  and  data  sharing  during 
SE  development.  The  collaborative  environment  tools  will  be  used  to  assist  in  the  definition, 
development,  coordination,  deployment,  use  and  maintenance  of  modeling  and  simulation  (M&S) 
capabilities.  To  support  these  objectives,  end-user  requirements  must  be  sufficiently  defined, 
categorized  and  compared  to  various  available  collaborative  planning  tools.  Additionally,  these 
requirements  should  be  prioritized  (if  practical)  and  used  to  develop  a  series  of  evaluation  metrics 
that  could  be  employed  to  characterize  sufficiency  of  the  various  tools  in  achieving  end-user 
requirements. 

1.1  Objectives 

The  objectives  of  this  work  are  two-fold: 

•  Survey  the  marketplace  for  available  tools  that  may  provide  collaborative  environments 
to  support  SE-based  exercises  and  experiments,  and  evaluate  the  most  relevant 
candidates; 

•  Develop  an  evaluation  method  for  assessing  collaborative  planning  and  engineering  tools. 

These  objectives  were  supported  by  a  number  of  other  discrete  activities,  such  as  literature 
review,  interviews,  and  prioritisation  of  requirements. 

1.2  Contract  Authority 

This  work  was  performed  under  contract  W7714-04900/001/SV.  The  Technical  Authority  for 
this  contract  is  Dr  Chris  Helleur.  The  Project  Authority  for  this  work  is  Dr  Wayne  Robbins,  and 
the  Scientific  Authority  for  this  work  is  Dr  Paul  Hubbard. 

1.3  Report  Organisation 

This  report  first  includes  a  brief  introduction  to  the  outcome  of  this  project  (called  the  Evaluation 
Matrix).  Detailed  methodology  information  about  the  conduction  of  a  literature  review  and 
subject  matter  expert  (SME)  interviews  follows.  Next,  the  development  of  the  evaluation  tool  is 
described.  Then,  the  reader  is  provided  with  instructions  about  how  to  use  the  evaluation  tool, 
including  five  worked  examples  (representing  the  most  common  and  most  likely  collaborative 
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environments  to  be  used  by  FFSE).  This  section  also  explains  how  to  use  the  tool  to  identify 
R&D  needs.  Finally,  conclusions  and  recommendations  are  made.  These  include  outlining  the 
most  appropriate  collaborative  environment  reviewed. 
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2  Brief  Introduction  to  the  Evaluation  Matrix 


A  brief  summary  of  the  outcome  of  the  project  (the  evaluation  matrix)  is  presented  in  this  section. 
It  is  thought  that  by  providing  this  information  up  front,  the  detailed  project  particulars  will  be 
more  easily  understood. 

The  general  principle  of  this  work  was  that  users  of  collaborative  planning  and  engineering  tools 
would  have  specific  uses  for  the  tools  or  goals  for  the  tools  known  prior  to  tool  selection  (e.g. 
Communication  and/or  Design).  Further,  specific  requirements  would  be  desired  of  the  tools 
(e.g.  Usable)  where  only  a  subset  of  requirements  would  be  applicable  to  each  use/goal.  Thus  the 
main  thrust  of  the  project  was  to  identify  these  ‘Uses/Goals’  and  ‘Requirements’.  The  team 
attempted  to  maximise  the  orthogonality  between  the  uses/goals  and  the  requirements;  that  is, 
they  attempted  to  ensure  that  the  uses/goals  and  requirements  were  as  distinct  from  each  other  as 
possible. 

This  led  to  the  creation  of  the  Evaluation  Matrix  with  the  ‘Uses/Goals’  down  the  left  (vertical 
axis)  and  the  ‘Requirements’  along  the  top  (horizontal  axis)  shown  below  in  Figure  1.  Particulars 
about  the  creation  of  the  matrix,  definitions  of  the  terms  in  the  matrix,  the  application  of  the 
matrix  in  tool  evaluation,  and  further  uses  for  the  Evaluation  Matrix  can  be  found  in  the  following 
sections  of  the  report. 
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Figure  1:  Evaluation  Matrix 
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3  Literature  Review 


The  first  step  in  the  project  was  to  conduct  a  literature  review.  We  developed  a  set  of  keywords 
(see  Table  1)  for  the  literature  search  based  on  our  experience  with  the  pertinent 
scientific/psychological,  human  factors,  and  military  domains  during  a  brainstorming  session  with 
all  members  of  the  literature  review  team.  These  keywords  were  chosen  because  they  focused  the 
search  on  topics  directly  related  to  distributed  team  collaboration  activity  and  were  intended  to  be 
able  to  identify  any  other  related  theoretical  approaches  or  conceptualizations  that  might  be 
relevant. 


Table  1:  Keywords  Used  for  Literature  Search 


Core 

Concept 

Primary  Keywords 

Related  Keywords 

Distributed 

Simulat*,  Technic*,  Collab*,  Environment 

Problem,  Tools,  Coordinat* 

Collaborat* 

Distributed,  Coordinat*,  Decision  Mak*, 
Problem  Solv*,  Computer  Supported 
Collaborative  Work  (CSCW),  Online 

FEDEP 

FEDEP,  SEDEP 

The  primary  keywords  were  the  most  important  words  used  in  the  search,  as  they  represented  the 
broad  relevant  constructs  likely  to  be  of  importance  in  research.  The  primary  keywords  were 
used  to  ensure  sampling  of  literature  from  several  different  areas  within  the  core  construct,  and 
their  use  was  guided  by  what  emerged  from  the  core  concepts.  For  example,  when  thinking  about 
the  concept  of  “collaboration”,  primary  keywords  such  as  “distributed”,  “coordinate”,  and 
“online”  emerged.  Related  keywords  were  used  when  the  primary  keyword  search  resulted  in  too 
many  potential  articles.  Related  keywords  would  narrow  the  search  within  the  output. 

The  asterisk  (*)  was  used  as  a  special  character  to  represents  any  other  characters,  so  as  to  not 
limit  search  terms.  For  example,  a  search  for  “coordinat*”  includes  searching  for  coordinate, 
coordination,  coordinates,  coordinating,  and  any  other  term  that  begins  with  “coordinat”. 

3.1  Databases 

The  primary  databases  were  the  most  relevant  for  searching  the  scientific/academic  literature  are 
outlined  in  Table  2  below. 

The  literature  review  thus  focused  on  finding  the  reported  ‘uses/goals’  of  collaborative  planning 
and  engineering  tools,  as  well  as  the  associated  ‘requirements’  of  users.  The  reader  should  note 
that  requirements  are  focused  on  supporting  human  interaction  and  not  detailed  technical  issues. 

Documents  were  selected  for  review  on  the  basis  of  their  relevance  to  distributed  collaborative 
planning  and  engineering.  This  immediately  rendered  many  documents  less  relevant  as  they 
focused  on  face-to-face  collaboration  or  collaboration  for  other  purposes. 
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Table  2:  Primary  Databases  for  Scientific/Academic  Search 


Database 

Description 

PsycINFO 

The  PsycINFO  database  is  a  collection  of  electronically  stored  bibliographic 
references,  often  with  abstracts  or  summaries,  to  psychological  literature  from  the 

1800s  to  the  present.  The  available  literature  includes  material  published  in  50 
countries,  but  is  all  presented  in  English.  Books  and  chapters  published  worldwide  are 
also  covered  in  the  database,  as  well  as  technical  reports  and  dissertations  from  the 
last  several  decades.  A  login/membership  is  required  to  access. 
http://www.apa.org/psycinfo/ 

HFES 

Human  Factors  and  Ergonomics  Society’s  mission  is  to  promote  the  discovery  and 
exchange  of  knowledge  concerning  the  characteristics  of  human  beings  that  are 
applicable  to  the  design  of  systems  and  devices  of  all  kinds.  A  login/membership  is 
required  to  access  the  articles,  http://www.hfes.org/ 

WWW 

A  general  search  of  the  World  Wide  Web,  mainly  through  Google  and  Google 

Scholar,  http://www.google.ca 

3.2  Identifying  ‘Uses/Goals’ 

Each  document  selected  for  review  was  considered  for  what  the  users  were  trying  to  do  via 
collaborative  means.  For  example,  many  papers  dealt  with  shared  document  editing.  Others 
dealt  with  the  problem  of  collaborative  planning  and  subsequent  sharing  of  planning  information 
to  remote  members  of  the  team.  This  provided  an  initial  list  of  ‘uses/goals’  for  consideration. 

In  some  senses,  identifying  the  reason  for  using  a  tool  was  the  easiest  part  of  the  task,  as  it  was 
usually  clear  what  the  authors  were  attempting  to  achieve.  This  information  was  generally 
provided  in  a  high  level  introduction  to  the  paper  intended  to  provide  context  for  the  reader. 

Identified  uses/goals  were  considered  for  their  relationship  to  each  other  and  to  requirements. 

3.3  Identifying  ‘Requirements’ 

Identifying  the  ‘requirements’  for  collaborative  planning  and  engineering  tools  proved  more 
difficult.  Usually,  outlining  requirements  was  not  the  primary  purpose  of  the  literature  reviewed 
and  the  reader  was  forced  to  ‘abstract’  requirements  from  authors’  discussions. 

General  rules  for  identifying  requirements  were  developed  and  used  by  members  of  the  project 
team.  These  rules  were: 

•  A  requirement  must  facilitate  goal  achievement; 

•  A  requirement  must  be  desirable  on  the  part  of  the  user;  and 

•  Requirements  need  not  be  specific  to  a  single  use/goal,  but  must  relate  to  at  least  one 
use/goal. 

The  initial  identification  of  requirements  was  not  restricted  (beyond  the  rules  above)  so  many 
requirements  overlapped  with  each  other,  and  with  the  uses/goals.  This  meant  that,  when 
developing  and  trialling  the  evaluation  matrix,  many  requirements  were  combined  or  eliminated. 
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3.4  Literature  Found  and  Reviewed 


A  large  number  of  papers  (greater  than  60)  were  reviewed  for  this  project.  However,  after 
reviewing  the  papers,  it  was  decided  that  only  35  were  directly  relevant  to  this  work. 

Accordingly,  these  35  papers  are  presented  in  Annex  A.  Due  to  the  fact  that  the  number  of  papers 
reviewed  exceeded  the  number  for  which  was  budgeted,  we  focused  our  review  only  on  the 
uses/goals  outlined  in  the  paper,  the  requirements  described  by  the  paper  and  the  tools  reported 
therein. 

3.5  Tools  Identified 

Similar  to  the  search  for  uses/goals,  each  document  selected  for  review  was  considered  for  the 
tools  described  therein  that  supported  collaborative  planning  and  engineering.  Often  these  might 
be  a  loose  collection  of  common  business  applications  such  as  e-mail  and  word  processors. 
However,  there  were  also  a  number  of  specifically-designed  tools  which  exist  specifically  to 
support  collaborative  work  between  geographically  distributed  teams.  This  provided  an  initial  list 
of  tools. 

The  list  of  tools  was  then  added  to  using  the  Internet  Google  search  engine  looking  for  tools  to 
support  collaborative  planning  and  engineering  functions.  This  search  in  particular  used  the 
terms  ‘FEDEP’  and  ‘SEDEP’.  This  led  to  discovering  the  richest  source  of  tools  and  tool 
descriptions  that  appear  on  the  resulting  list,  which  was  the  Wikipedia  online  encyclopaedia 
(http://www.wikipedia.org/wiki/List_of_collaborative_software). 

A  total  of  215  tools  were  uncovered  during  the  course  of  this  project.  The  tools  are  listed  and 
described  in  Annex  B,  along  with  information  about  whether  they  support  the  FEDEP  and 
whether  they  are  open  source. 

A  number  of  tools  were  identified  that  were  relevant  to  the  FEDEP.  However,  most  collaborative 
tools  are  focused  on  collaboration  for  crisis  management.  Further  to  this,  a  large  pan-European 
project  (EUCLID)  was  focused  on  the  FEDEP  specifically,  but  took  a  process-view,  rather  than  a 
collaborative-tool-support-view.  As  such,  though  effective  EUCLID  tools  were  developed,  they 
focused  on  the  technical  side,  and  did  not  necessarily  include  how  goals  of  users  interacted  with 
tool  technology.  As  such,  we  feel  that  the  aim  of  the  current  project,  to  investigate  collaborative 
tools  specifically  for  planning  and  engineering  of  distributed  simulation  activities,  is  timely  and  is 
not  replicated  elsewhere  in  the  world. 
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4  Subject  Matter  Expert  (SME)  Interviews 


Immediately  following  the  literature  review,  Subject  Matter  Experts  (SMEs)  were  interviewed  for 
the  project.  At  the  start-up  meeting  with  the  Project  and  the  Scientific  Authorities,  stakeholder 
groups  that  should  be  interviewed  and  the  specific  points  of  contact  within  these  groups  were 
identified.  This  list  represented  contacts  already  held  by  the  Project  and  Scientific  Authorities,  as 
well  as  those  suggested  by  the  contractor.  A  total  of  12  interviews  were  conducted. 

4.1  Interview  Participants 

The  interview  subjects  and  their  affiliations  are  as  follows: 

•  Dr  Phil  Farrell  -  Canadian  Forces  Experimentation  Centre  (CFEC) 

•  Bob  Elliot  -  Head  of  Synthetic  Environment  Coordination  Centre  (SECO) 

•  Capt  Pete  Dietert,  Capt  Ray  Dean  &  Mr  Bob  Kirk  -  DAR  7-3-2 

•  Major  Scott  Arbuthnott  -  Synthetic  Environment  Coordination  Office  (SECO),  CF  Air 
Warfare  Centre 

•  Kendall  Wheaton  -  Canadian  Forces  Experimentation  Centre  (CFEC) 

•  Warner  Montiero  -  WaveRate  Communications  Corporation 

•  Doug  Brown  and  Norm  Green  -  Army  Simulation  Centre  (Calian  Contractors) 

•  Mike  Lepard  -  Synthetic  Environment  Research  Facility,  DRDC  Toronto  (CAE 
Professional  Services  Contractor) 

•  Dr  Rick  Bodner  -  Simulation  and  Modelling  for  Acquisition,  Rehearsal  and  Training 
(SMART)  Section,  DRDC  Toronto 

Eleven  interviews  were  conducted  face-to-face  following  the  semi-structured  interview  format 
outlined  below.  One  interview  was  conducted  by  email. 

4.2  Standard  Questions 

The  semi-structured  interview  centred  on  a  number  of  ‘standard’  (i.e.  asked  of  every  participant) 
questions.  These  questions  were  selected  to  facilitate  responses  from  Subject  Matter  Experts 
(SMEs)  if  they  were  not  available  to  meet  and  be  interviewed.  However,  when  a  face-to-face 
interview  took  place,  the  conversation  was  permitted  to  range  freely,  and  the  interviewer  made 
sure  to  get  the  critical  information  outlined  in  the  standard  questions. 

The  standard  questions  were: 

•  What  tasks  associated  with  preparing/developing  a  distributed  simulation  do  you 
currently,  or  would  you  find  useful  to,  perform  collaboratively  with  a  distributed/remote 
team? 
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HUMAN.' SYSTEMS 


•  What  type  of  tool  (general  functionality  or  specific  brand  name)  do  you  feel  would 
enhance  your  ability  to  perform  these  tasks? 

•  In  working  collaboratively  with  a  distributed  team  to  stage  a  distributed  simulation,  what 
problems  do  you  typically  encounter? 

•  What  tools  do  you  find  useful  in  eliminating  problems?  (e.g.  software,  phone,  e-mail, 
etc.) 

•  What  tools  exist  that  you  think  would  be  useful  enhancements  to  your  ability  to 
collaborate  with  remote  team  members? 

•  What  capabilities  do  you  think  need  to  be  fielded  to  enhance  your  ability  to  collaborate 
with  remote  team  members? 

•  For  each  capability  that  you  have  identified,  what  are  the  specific  requirements  associated 
with  that  capability? 

•  With  specific  reference  to  the  FEDEP,  what  problems  do  you  find  when  your  distributed 
team  attempts  to  run  the  FEDEP? 

•  How  could  these  problems  with  running  the  FEDEP  be  overcome? 

Interviews  were  recorded  (with  participants’  permission)  and  key  points  were  summarized  for 
inclusion  in  this  report  (the  next  section).  Note  that  no  new  tools  were  mentioned  during  the 
interviews  that  were  not  already  included  on  the  Tool  List  (Annex  B). 


4.3  Results  of  the  SME  Interviews 

The  following  sections  outline  a  general  summary  of  what  was  stated  during  the  interviews.  Note 
that  the  statements  in  the  rest  of  this  section  are  only  the  opinions  of  individuals  interviewed. 
These  views  have  not  been  verified  for  accuracy,  nor  are  the  statements  necessarily  shared  by  all 
who  were  interviewed. 

4.3.1  SMEs:  General  Comments 

There  are  a  number  of  requirements  that  are  very  specific  to  DND  applications.  In  particular, 
network  access  may  be  problematic  for  some  individuals  because  of  the  classification  of  some 
networks  and  the  location  of  some  individuals.  Additionally,  there  is  a  need  for  virtually  all  DND 
websites  to  be  bilingual,  which  may  impose  significant  budget  implications  of  a  collaborative 
environment. 

The  bulleted  list  below  outlines  specific  collaboration  uses/solutions  mentioned  by  those 
interviewed: 

•  A  Wiki1  was  proposed  as  an  effective  form  of  collaboration  online,  but  there  are  a 
number  of  technical  barriers  to  overcome.  Those  with  experience  with  Wiki’s  find  that 


1  “Wiki”:  A  website  or  similar  online  resource  which  allows  users  to  add  and  edit  content  collectively. 
www.parliament.vic.gov.au/sarc/E-Democracy/Final  Report/Glossarv.htm 
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they  facilitate  the  sharing  and  editing  of  information  very  effectively.  However,  others 
with  limited  technical  experience  may  be  initially  uncomfortable  with  adding  information 
or  editing  information  currently  on  the  Wiki.  With  proper  training,  these  initial 
apprehensions  can  be  overcome.  Those  interviewed  had  used  Wiki’s  for  technical  lists, 
but  not  for  documents  or  more  tangible  instantiations  of  simulation  elements. 

•  The  ability  to  playback  a  data  logger  while  discussing  the  data  would  be  good.  This 
would  also  be  useful  in  a  VTC  environment. 

•  Coordination  of  resources  is  difficult.  A  shared  calendar  is  one  way  of  overcoming  this, 
by  showing  people’s  availability  to  meet  and  equipment/resource  availability  for  use. 

•  Bandwidth  issues  were  important.  Sharing  a  large  PowerPoint  file  over  a  network 
(especially  at  the  beginning  of  or  during  a  VTC)  imposes  unacceptable  delays.  However, 
this  is  in  the  absence  of  a  common  server,  which,  if  everyone  could  access  the  server, 
would  allow  much  greater  sharing  and  reduce  delays. 

•  Perhaps  the  biggest  barrier  to  tool  usage  is  cultural.  People  need  to  have  a  positive 
experience  otherwise  they  will  not  use  a  tool  again.  Though  technology  staff  tend  to  be 
more  adventurous  in  trying  out  a  new  tools,  they  can  also  be  more  uncompromising  if 
something  does  not  work  as  expected. 

•  Tools  need  to  be  open-source.  This  allows  the  technologists  involved  in  the  work  to 
modify  them  to  suit  the  specific  needs  of  the  project.  Although  this  is  likely  to  be 
somewhat  uncoordinated,  it  means  that  collaborative  tools  for  planning  and  engineering 
will  be  constantly  evolving  to  be  relevant  to  the  task  at  hand. 

4.3.2  SMEs:  FEDEP  Steps  and  Collaboration 

The  FEDEP  process  was  roughly  followed  by  those  interviewed;  however,  some  sub-steps  were 
perceived  to  be  not  required  and  consequently  were  skipped.  FEDEP  steps  are  not  serial,  and 
users  can  move  onto  the  next  step  while  the  previous  step  is  not  yet  complete  (though  a  user 
should  not  be  more  than  2  steps  away  from  the  incomplete  step).  The  following  describes  some 
perceptions  of  the  FEDEP  expressed  during  the  interviews  and  how  collaboration  evolved 
throughout  the  process. 

4. 3. 2.1  Step  1  -  Define  Federation  Objectives 

This  step  was  done  by  the  project  sponsor  when  the  goals  of  the  project  were  initially  outlined. 

4. 3. 2.2  Step  2  -  Develop  Federation  Conceptual  Model 

The  scenario  and  agreements  made  were  all  written  and  outlined  in  the  Synthetic  Environment 
Development  Plan  (SEDP).  This  included  a  complete  description  of  scenarios,  and  so  forth,  at  a 
high  level;  technologists  must  then  figure  out  how  to  achieve  what  is  in  the  SEDP.  The  SEDP 
was  effectively  the  first  collaboration  tool:  a  word  document  describing  what  everyone  should  be 
working  toward. 

The  SEDP  was  also  useful  to  pass  onto  new  staff  to  orient  them,  and  others  interested  in  joining, 
as  it  clearly  outlined  what  was  required.  It  was  also  a  useful  tool  to  ensure  that  people  are 
committed  to  something  in  full  knowledge  of  what  that  something  is  and  how  it  fits  into  the 
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broader  concept.  The  SEDP  was  compact  at  approximately  30  pages  in  4  sections.  The  SEDP 
was  e-mailed  and  subject  to  a  VTC  discussion  and  quarterly  face-to-face  meetings.  Online 
editing  of  this  document  would  have  been  useful  to  facilitate  these  discussions.  Also  at  this  stage, 
roles  and  working  groups  were  set  up. 

The  Federation  Exercise  Planning  Workbook  (FEPW)  is  similar  to  SEDP,  though  slightly  more 
expansive.  It  is  basically  a  project  folder/plan. 

4. 3.2.3  S  tep  3  -  Design  Fed  era  tion 

Step  3  included  many  meetings,  including  quarterly  meetings.  The  SEDP  was  continually 
referred  to  for  the  purposes  of  guiding  design. 

Meetings  were  structured  so  that  everyone  got  together  and  focused  on  objectives.  Then  people 
were  split  into  working  groups  (6-8  people)  based  on  the  objectives  and  steps  in  SEDP. 
Typically  after  a  day  and  a  half,  the  working  groups  reported  on  progress  made  and  decided  on 
the  actions  to  be  achieved  before  the  next  meeting.  A  great  deal  of  progress  was  made  at 
meetings,  and  very  little  was  done  in  between.  Occasionally  there  was  a  person  working  on  a 
part,  but  nothing  particularly  substantial. 

The  distributed  project  management  aspect  was  challenging,  especially  as  the  promise  that 
actions  will  be  reviewed  at  the  next  meeting  did  not  provide  an  incentive  to  work.  The  challenge 
was  amplified  if  the  project  manager  had  no  direct  authority  over  a  group/individual.  As  such, 
completing  the  exercise  was  largely  about  personal  relationships  and  individual  motivation  to  do 
work. 

There  are  also  significant  differences  in  culture  in  terms  of  authority  and  choice: 

•  Army  -  Need  higher  approval  to  do  the  work.  Once  approved,  will  then  do  it  no  matter  if 
low  personal  interest. 

•  Navy  -  Need  approval  by  immediate  superior  to  work. 

•  Air  Force  -  There  is  more  autonomy  in  deciding  what  work  to  do. 

•  DRDC  -  May  have  approval  and  still  may  not  do  the  work  if  not  personally  interested. 

4. 3. 2.4  Step  4  -  Develop  Federation 

The  team  eventually  got  the  Federation  Object  Model  (FOM),  Federation  Agreement  Document 
(FAD)  and  Runtime  Infrastructure  (RTI)  software  and  hardware,  but  the  process  (via  request  for 
proposal)  took  approximately  6  months.  The  requirements  document  developed  at  this  stage  was 
short,  but  the  words  were  defined  carefully,  requiring  much  collaboration  and  revision.  Once  the 
software  and  hardware  was  obtained,  FEDEP  Step  4  began  in  earnest.  At  this  point,  the  process 
became  quicker  because  everyone  who  needed  tools  had  them. 

A  great  deal  of  collaboration  was  needed  to  get  equipment  to  talk  to  each  other.  However,  this 
could  be  derailed  by  official  requests  for  new  networks  and  decisions  about  the  classification 
level  of  the  networks.  This  led  to  the  conclusion  that  both  a  classified  and  unclassified 
development  environment  was  needed.  SharePoint  was  used  as  an  unclassified  environment 
collaboration  tool,  while  Wikis  and  Internet  Protocol  (IP)  phones  were  used  in  the  classified 
environments.  There  was  a  concern  about  the  use  of  SharePoint  as  the  tool  was  not  guaranteed  to 
be  maintained,  however,  no  alternatives  were  found  and  so  the  tool  continued  to  be  used. 
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Another  critical  collaboration  requirement  at  this  stage  of  the  FEDEP  was  for  a  contact  list  to 
know  who  was  doing  what  and  how  to  contact  them. 

4. 3. 2.5  Step  5  -  Integrate  and  Test  Federation 

Wiki  pages  were  again  the  primary  means  of  coordinating  activities  and  content.  Data  could  not 
be  sent  over  the  network  until  a  security  certificate  was  obtained,  which  took  a  great  deal  of  time. 
In  general,  the  security  considerations  of  following  the  FEDEP  for  DND  purposes  added 
significant  time  to  the  process;  even  clearances  to  perform  tests  were  not  received  until  hours 
before  the  event. 

An  ideal  collaborative  tool  would  allow  the  testing  of  federates  as  they  are  being  constructed. 

This  would  include  good  loggers  and  filters.  A  nice  to  have  would  have  also  been  a  network 
management  tool  to  tell  you  whether  the  network  is  awake  on  the  other  side  and  also  to  manage 
(or  at  least  measure)  bandwidth  issues  (e.g.  who’s  using  what,  so  that  issues  can  be  sorted  out 
over  the  telephone). 

4. 3. 2.6  Step  6  -  Execute  Federation  and  Prepare  Outputs 

During  execution,  the  only  collaborative  planning  and  engineering  tools  that  were  considered  for 
use  were  video  conferencing  facilities.  However,  this  was  not  a  classified  resource  and  would 
thus  only  have  been  used  for  briefing  and  debriefing.  Consequently,  they  were  not  used.  IP 
Phones  continued  to  be  used  throughout  execution  for  technical  discussions. 

4. 3. 2.7  Step  7  -  Analyse  Data  and  Evaluate  Results 

The  lessons  learned  from  the  project,  including  those  pertaining  to  the  FEDEP  were  posted  to  a 
computer’s  hard  drive,  but  archiving  is  still  being  done,  as  is  the  taking  of  corrective  actions. 

A  further  deficiency  in  support  to  the  FEDEP  is  the  lack  of  something  to  share  conceptual  models 
early  in  the  process.  Typically  this  might  be  Visio  drawings  or  Mind  Manager  maps.  Mind 
Manager  in  particular  is  useful  for  developing  class  networks  which  can  then  be  easily  shared 
between  different  people  and  groups.  If  nothing  else,  there  should  have  at  least  have  been  a 
shared  repository  of  data,  especially  the  large  files  (e.g.  terrain). 

The  process  also  needs  Authoritative  Data  Sources  (i.e.  a  tag  to  a  file  that  indicates  it  is  the  one  to 
use)  and  Authoritative  Operational  References.  These  will  help  ensure  coordination  and  realism. 

One  of  the  primary  problems  is  that  people  do  not  actually  know  what  some  of  the  things  in  the 
FEDEP  are.  For  instance,  the  Federation  Object  Model  (FOM)  is  a  tree  structure  of  objects  in  the 
federation,  but  most  people  don’t  know  what  it  is.  The  FOM  is  meant  to  be  in  XML  and  should 
thus  be  readable  by  human  or  machine  to  describe  the  objects,  attributes,  interactions  and 
parameters  of  interactions.  The  FOM  is  important  because  it  decides  how  federates  should  work 
together  (mediated  by  the  RunTime  Infrastructure  (RTI)). 

Another  problem  with  the  FEDEP  is  the  configuration  of  documents.  The  FAD  defines 
agreements  on  how  to  set  machines  (e.g.  terrain  to  use,  coordinate  system,  algorithms,  etc.). 
FEDEP  use  in  Canadian  applications  lack  sufficient  use  of  the  FAD,  which  is  not  always  kept  up 
to  date.  It  can  be  a  critical  problem  when  people  are  not  following  the  FAD,  but  it  does  not 
actually  preclude  the  running  up  of  a  federation. 
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4. 3. 2. 8  Summary  of  FEDEP  Collaboration  Opportunities 

Table  3  below  summarizes  the  collaboration  opportunities  that  were  noted  for  each  step  of  the 
FEDEP  in  the  interviews. 


Table  3:  Summary  of  Collaboration  Opportunities 


FEDEP  Step 

Collaboration  Opportunities  Noted  During  Interviews 

1 

n/a 

2 

The  SEDP  was  a  word  document  describing  what  everyone  should  be  working  toward.  It  was  e- 
mailed  and  subject  to  a  VTC  discussion  and  quarterly  face-to-face  meetings.  Online  editing  of  this 
document  would  have  been  useful  to  facilitate  these  discussions. 

3 

Many  meetings  were  held,  including  quarterly  meetings.  The  SEDP  was  continually  referred  to  for 
the  purposes  of  guiding  design. 

The  distributed  project  management  aspect  was  challenging. 

4 

The  requirements  document  developed  at  this  stage  was  short,  but  the  words  were  defined 
carefully,  requiring  much  collaboration  and  revision. 

A  great  deal  of  collaboration  was  needed  to  get  equipment  to  talk  to  each  other,  which  was 
complicated  due  to  classification  level.  Eventually,  both  a  classified  and  unclassified  development 
environment  was  created:  SharePoint  was  used  as  an  unclassified  environment  collaboration  tool, 
while  Wikis  and  IP  phones  were  used  in  the  classified  environments. 

Another  critical  collaboration  requirement  at  this  stage  of  the  FEDEP  was  for  a  contact  list  to  know 
who  was  doing  what  and  how  to  contact  them. 

5 

Wiki  pages  were  again  the  primary  means  of  coordinating  activities  and  content. 

An  ideal  collaborative  tool  would  allow  the  testing  of  federates  as  they  are  being  constructed. 

Would  include  good  loggers  and  filters. 

A  network  management  tool  would  also  be  nice  to  have  that  would  tell  you  whether  the  network  is 
awake  on  the  other  side  and  that  would  also  manage  (or  at  least  measure)  bandwidth  issues  (e.g. 
who’s  using  what),  so  that  issues  can  be  sorted  out  over  the  telephone. 

6 

During  execution,  the  only  collaborative  planning  and  engineering  tools  that  were  considered  for 
use  were  video  conferencing  facilities.  However,  this  was  not  a  classified  resource  and  would  thus 
only  have  been  used  for  briefing  and  debriefing.  Consequently,  they  were  not  used. 

IP  Phones  continued  to  be  used  throughout  execution  for  technical  discussions. 

7 

The  sharing  of  conceptual  models  early  in  the  process  (e.g.  Visio  drawings,  or  Mind  Manager 

Maps)  was  desired.  At  minimal,  there  should  have  been  a  shared  repository  of  data,  especially  the 
large  files  (e.g.  terrain).  Ideally  this  is  required  to  be  provided  by  the  system  architect  so  that  all 
others  involved  understand  the  system  at  a  high  level. 

The  process  also  needs  Authoritative  Data  Sources  (i.e.  a  tag  to  a  file  that  indicates  it  is  the  one  to 
use)  and  Authoritative  Operational  References.  These  will  help  ensure  coordination  and  realism. 

The  FAD  defines  agreements  on  how  to  set  machines  and  needs  to  be  kept  up  to  date. 

Additional  comments  made  during  the  interviews  that  were  unrelated  to  the  FEDEP,  but  related 
to  collaboration  in  general  have  been  provided  to  the  project  authorities  separate  from  this  report. 
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5  The  Evaluation  Matrix 


This  section  details  the  development  of  the  Evaluation  Matrix.  It  also  explains  the  terms  within  the 
matrix,  and  outlines  different  applications  for  it. 

5.1  Development  of  the  Evaluation  Matrix 

The  evaluation  matrix  was  intended  to  serve  a  number  of  functions: 

•  Evaluate  tools  for  collaborative  planning  and  engineering,  based  on  satisfaction  of 
requirements; 

•  Identify  which  tools  are  most  adequate  across  a  variety  of  requirements; 

•  Identify  which  tools  are  most  adequate  for  specific  uses  or  work  goals; 

•  Identify  research  needs  for  such  tools; 

•  Assist  in  the  selection  of  tools  to  support  collaborative  planning  and  engineering;  and, 

•  Guide  the  development  of  new  tools  to  support  collaborative  planning  and  engineering. 

To  achieve  this,  the  evaluation  matrix  must  represent  the  requirements  for  a  tool  to  support 
collaborative  planning  and  engineering  along  one  axis,  and  the  uses/goals  of  such  tool  use  along 
the  other  axis.  These  two  axes  need  to  be  as  independent  as  possible,  which  is  referred  to  as 
‘orthoganality’  throughout  the  rest  of  the  report.  When  creating  each  of  the  different  axes,  effort 
was  placed  on  ensuring  that  if  a  tool  supported  a  particular  use  it  would  not  unintentionally  be  seen 
to  support  a  number  of  related  requirements.  However,  the  converse  need  also  not  be  true,  such 
that  if  a  tool  satisfied  a  requirement  it  did  not  necessarily  fulfill  a  use/goal. 

All  uses/goals  and  all  requirements  identified  from  the  literature  review  and  the  SME  interviews 
were  listed  on  two  axes.  Then  all  the  requirements  were  considered  for  overlap.  If  two 
requirements  were  deemed  to  refer  to  the  same  thing,  one  was  deleted  and  the  wording  of  the 
remaining  requirement  was  reconsidered  to  ensure  clarity.  If  two  requirements  were  similar,  but 
referred  to  slightly  different  issues,  the  combining  of  the  two  was  considered.  If  it  was  decided  to 
do  so,  the  wording  was  changed  to  ensure  that  the  resultant  requirement  accurately  reflected  the 
different  aspects. 

A  similar  process  was  undertaken  with  uses/goals.  If  two  uses/goals  were  deemed  identical,  one 
was  deleted,  and  if  two  uses/goals  were  similar  they  may  be  combined. 

Additionally,  further  separation  was  necessitated  between  uses/goals  and  requirements  as  there  was 
initially  some  overlap  between  the  two  categories.  Thus,  requirements  were  considered  against  the 
uses/goals,  and  uses/goals  were  considered  against  requirements.  If  a  use/goal  and  requirement 
were  perceived  to  be  similar,  a  decision  was  made  regarding  how  to  increase  the  orthogonality. 

This  could  be  done  by  reframing  the  use/goal  or  the  requirement  to  make  it  different,  or  by  deleting 
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one  or  the  other.  This  was  considered  on  a  case  by  case  basis  with  no  priority  given  to  one  axis 
over  the  other. 

Requirements  were  initially  grouped  together  with  other  related  requirements.  These  groups 
changed  as  various  steps  were  taken  to  rationalise  the  requirements  and  ensure  orthogonality  with 
the  uses/goals.  Requirements  were  also  prioritised  on  a  scale  of  1  -  5  (5  being  a  high  priority;  1 
being  a  low  priority).  The  prioritisation  was  then  used  to  develop  a  weighted  score  to  allow  users 
of  the  evaluation  matrix  to  quickly  determine  what  tool  was  best  suited  overall,  and/or  for  specific 
uses/goals. 

However,  it  was  also  recognised  that  most  requirements  were  only  applicable  to  a  specific  few 
uses/goals.  Accordingly,  each  requirement  was  considered  for  its  applicability  for  each  use/goal. 

If  it  was  determined  that  the  requirement  was  not  applicable,  the  corresponding  cell  in  the 
evaluation  matrix  was  shaded.  This  meant  that  the  weighted  score  must  accommodate  the  fact  that 
some  requirement  scores  would  be  multiplied  by  a  reduced  number  of  uses/goals,  while  other 
requirement  scores  would  be  multiplied  by  the  total  number  of  uses/goals.  The  formula  that 
calculates  the  score  reflects  these  factors. 

The  user  of  the  evaluation  matrix  is  therefore  only  required  to  evaluate  some  requirements  for  some 
uses/goals.  The  user  can  assess  the  requirement  as: 

•  Satisfied  -  the  tool  is  fully  capable  of  achieving  the  ‘use/goaT  as  defined  by  the 
‘requirement’; 

•  Partially  satisfied  -  the  tool  is  only  partially  capable  of  achieving  the  ‘use/goaT  as  defined 
by  the  ‘requirement’; 

•  Not  satisfied  -  the  tool  is  incapable  of  achieving  the  ‘use/goaT  as  defined  by  the 
‘requirement’;  and, 

•  Unknown  -  the  capability  of  achieving  the  ‘use/goaT  as  defined  by  the  ‘requirement’  is 
unknown. 

To  assist  the  user  in  appreciating  the  general  trend  of  the  tool  in  meeting  requirements  and 
uses/goals,  the  evaluation  matrix  was  colour  coded  according  to  the  assessments  above. 

•  Satisfied  -  green 

•  Partially  satisfied  -  white 

•  Not  satisfied  -  red 

•  Unknown  -  yellow 

Some  requirements  are  more  general  and  applicable  to  all  uses/goals.  These  tended  to  be 
requirements  such  as  usability,  improve  productivity,  etc.  In  this  case,  the  requirement  was 
assessed  for  every  use/goal  and  cannot  be  applied  universally.  As  such,  the  user  must  still  consider 
every  requirement  discretely  for  each  use/goal. 
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HUMANS  YS  TEMS 


Having  constructed  the  evaluation  matrix,  it  was  then  necessary  to  trial  its  use.  This  led  to  further 
reductions  of  requirements  and  uses/goals.  These  reductions  continued  to  be  based  on 
repetition/overlap  and  orthogonality,  but  were  also  now  made  on  the  basis  of  the  utility  of  the 
requirement  or  use/goal.  Specifically,  a  requirement  must  be  perceived  to  assist  in  determining 
what  collaborative  planning  and  engineering  tool  is  best,  or  where  the  gaps  in  collaborative 
capability  exist.  In  the  course  of  a  number  of  trials  of  the  evaluation  matrix,  the  concept  of  utility 
led  to  the  reduction  in  the  number  of  requirements  used  for  evaluation.  No  uses/goals  were 
discarded  at  this  stage  of  development. 

Due  to  the  number  of  potential  tools  uncovered  during  the  course  of  this  review,  it  was  not  possible 
to  perform  an  evaluation  of  all  tools  (see  full  list  of  tools  in  Annex  B).  Instead,  only  those 
available  tools  mentioned  in  more  than  one  interview  were  evaluated.  These  tools  are  outlined 
below  in  Table  4. 


Table  4:  Tools  Evaluated 


Name 

Brief  Description 

Provide 

SharePoint 

An  integrated  suite  of  server  capabilities  for  enterprise  search,  content  management, 
business  process  facilitation,  simplified  information  sharing,  and  enhanced  business 
insight 

Microsoft 

NetMeeting 

VoIP  and  multi-point  videoconferencing  client 

Microsoft 

LiveLink 

Web  content  management  solution  that  enables  organizations  to  create,  search  and 
manage  content. 

Open  Text 

Groove 

Custom  peer  to  peer  application  for  distributed  teams  that  enables  the  creation  of 
collaboration  workspaces 

Microsoft 

Google  Docs 

Web-based  word  processor  and  spreadsheet  application  that  allows  users  to  create 
and  edit  documents  and  spreadsheets  online  while  collaborating  in  real-time  with 
other  users 

Google 

5.2  Description  of  Matrix  Components 

The  evaluation  matrix  is  comprised  of  19  uses/goals  (see  Table  5)  and  55  requirements  (see  Table 
6).  They  are  ordered  in  the  table  below  in  terms  of  relatedness.  The  various  priorities  of  the 
requirements  (5  being  highest  priority;  1  being  lowest  priority)  are  provided  in  Table  6.  Priorities 
are  based  on  statements  collectively  made  in  the  SME  interviews. 


Table  5:  Definitions  of  the  Uses/Goals 


Use/Goal 

Definition 

Communication 

Exchange  of  info  in  real  time  via  voice,  text  or  other  means 

Design 

Development  of  ideas  describing  how  a  product  should  look,  act,  etc. 

File  sharing 

Ability  for  two  or  more  people  to  use  a  file 

File  storage 

Ability  to  store  a  file  somewhere  where  all  nominated  users  can 
access  it 
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Use/Goal 

Definition 

Archiving 

Automatic  storage  of  files  as  they  change  so  that  it  is  possible  to  revert 
to  an  earlier  version  of  a  file  if  necessary 

Simulation/demonstration 

Ability  to  run  the  product  so  that  local  and  remote  users  can  see  it 
working  and  participate/interact  with  it 

Testing/validation 

Ability  for  a  product  to  be  tested  by  local  and  remote  users 

Planning 

Ability  for  local  and  remote  users  to  collaborate  to  develop  an  accurate 
and  workable  plan 

Project  monitoring/control 

Ability  for  nominated  individual(s)  to  determine  at  any  time  how  the 
project  is  progressing  and  exercise  control  measures 

Workflow  control 

Automatic  function  by  which  the  system  determines  project  progress 
and  suggests  next  work  item  to  the  next  person  who  logs  in  to  the 
system 

Security/access  control 

Ability  to  restrict  access  to  approved  individuals 

Social  activities  and  team  building 

Provision  of  avenues  to  develop  and  support  team  connections 
beyond  strictly  project  related  connections  (involves  the  inherent 
recognition  that  these  additional  connections  are  a  necessary  part  of  a 
high  performing  team) 

Automated  project  support 

Provision  of  statistics  and  reminders  that  assist  project  team  members 
to  carry  out  the  process  assigned  and  to  do  so  in  a  timely  and  effective 
manner 

Resource  allocation 

Automatic  process  by  which  the  system  decides  who  has  the  required 
skills  and  assigns  new  or  unfinished  tasks  accordingly  and/or 
determines  who  has  finished  assigned  tasks 

Decision  support 

Automatic  function  by  which  the  system  provides  data  and 
suggestions  to  support  the  project  related  decision  making  by  the 
project  team 

Analysis  support 

Suite  of  functions  that  the  project  team  can  use  or  direct  to  derive 
meaningful  data  about  the  process  or  product 

Team  coordination 

Based  on  the  plan  (schedule  and  assignment)  will  e-mail  reminders  to 
team  members,  remove  tasks  from  assignment  when  complete, 
initiate  new  tasks,  show  who  is  involved  on  same  and  other  tasks,  and 
show  relationship  between  tasks 

Software  coding/programming 

Create  software  to  achieve  some  desired  outcome 

Network  development 

Linking  technologies  so  that  separate  hardware  can  ‘talk’  to  each  other 

Table  6:  Definitions  and  Priorities  of  the  Requirements 


Requirement 

Priority 

Definition 

Supports  synchronous  work  and 
communications 

5 

Team  members  can  work  during  same  time  periods 
to  share  and  communicate  with  each  other 

Supports  asynchronous  work  and 
communications 

5 

Team  members  do  not  need  to  be  available  at  the 
same  time  to  work  or  communicate 

Provides  real-time  synchronisation  of 
views,  including  user  interactions 

5 

Remote  users  can  see  what  other  remote  users  are 
doing  to  a  file/application  as  it  is  done 

Fast,  consistent  response  and  transfer 
times 

4 

No  appreciable  delay  to  system  response  time,  in 
spite  of  using  a  network 
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Requirement 

Priority 

Definition 

Graphic  interface  and  mark-up 
functionality 

4 

Changes  should  be  indicated  on  screen  by  some  sort 
of  colour  or  symbolic  coding 

Supports  distributed  teams 

1 

Supports  teams  with  members  who  are  not  in  the 
same  room,  building,  city,  country,  etc. 

Has  distinct  member  roles 
(active/passive,  master/client) 

2 

Aids  coordination  of  activities  between  team 
members,  to  avoid  confusion  and  error 

Searchable/writable  database  access 
with  reusable  classes  (federates, 
federations,  etc.) 

1 

For  FEDEP,  team  members  should  be  able  to  write 
new  federates  to  a  database  for  future  reuse,  and 
others  should  be  able  to  search  for  entries  that  might 
save  time  and  effort  in  future  simulations 

Supports  heterogeneous  software  (inc 
non-collaborative  apps) 

1 

Allows  software  not  constructed  as  part  of  the 
collaborative  tool  to  be  incorporated  into  the  working 
process,  without  any  modification 

Error  Tolerant 

1 

If  there  is  a  loss  of  communications,  or  someone 
makes  an  error  in  their  entry,  the  tool  does  not  refuse 
to  function 

Allows  for  user  privacy 

1 

Permits  the  activities  and/or  objects  of  a  user  to  be 
hidden  from  other  users 

Little  user  training  required 

5 

The  manner  in  which  the  tool  is  used  should  be 
immediately  apparent  to  users 

Results  in  increased  productivity 

5 

Self  explanatory. 

Ability  to  lock'  object/file 

1 

Objects/files  should  be  rendered  uneditable  to  guard 
against  inadvertent  and  uncoordinated  changes  to 
critical  elements 

Accommodates  standard  policies  for  use, 
inc  processes,  IP,  etc. 

5 

Tool  should  allow  the  user(s)  to  determine  how  the 
tool  should  be  used  and  should  permit  rules  and 
restrictions  to  be  built  in  to  the  application 

Has  context  management 

3 

System  knows  what  the  user  is  trying  to  do  and 
presents  options  that  are  related  to  that  use 

Includes  software  agents  that  determine 
what  files/documents  are  impacted  by  a 
change 

4 

Helps  the  user  to  decide  what  else  needs  to  be  done 
subsequent  to  a  change 

Indicates  'new',  'changed' 

5 

Either  in  text,  colour  or  symbolically  (or  all) 

Maintains  single  version  of  document  w/ 
auto-version  control 

5 

To  save  space  and  bandwidth  a  single  version  is 
maintained,  but  version  control  (date,  time)  allows 
user  to  determine  whether  it  is  desirable  to  return  to 
an  earlier  version 

Indicates  Personnel  Presence 

5 

Indicate  whether  a  user  is  online 

Indicates  Network  Presence 

5 

Indicate  whether  another  network  is  available  for  use 

Minimizes  formatting 

4 

No/minimum  formatting  required  ensuring  no  time  lost 
making  documents  adhere  to  an  organisational 
standard  (especially  relevant  to  new  team  members) 

Templates  available 

4 

Templates  can  be  automatically  provided  to  ensure 
the  content  of  a  document/file/etc  is  clear 

Supports  word  processing  documents 

5 

Self  Explanatory 
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Requirement 

Priority 

Definition 

Supports  spreadsheet  documents 

5 

Self  Explanatory 

Supports  presentation  documents 

5 

Self  Explanatory 

Supports  software  code  files 

5 

Self  Explanatory 

Allows  application  sharing  &  shared  file 
editing 

5 

Allows  users  to  share  an  application  that  may  only  be 
resident  on  a  single  machine,  and  to  share  a  single 
version  of  a  file  that  may  only  be  resident  in  one  place 

Supports  text  communications  (including 
e-mail,  chat,  etc.) 

5 

Allows  or  provides  text  communications  within  the 
application 

Supports  audio  communications 

5 

Allows  or  provides  audio  communications  within  the 
application 

Supports  video  communications 

2 

Allows  or  provides  video  communications  within  the 
application 

Supports  heterogeneous 
communications 

2 

Allows  users  to  communicate  via  mixed  mode;  e.g. 
one  user  using  text  and  the  other  using  voice 

Open  Source 

5 

The  code  behind  the  application  should  be  accessible 
and  editable  by  anyone 

Application  trusted 

3 

Users  should  trust  the  application  to  do  what  they 
want  it  to  do 

Supports  top-down  working 

3 

Permits  the  user  to  get  an  overview  first  and  then 
obtain  detail  as  desired 

Supports  bottom-up  working 

3 

Permits  the  user  to  obtain  detail  first  and  use  this 
anyway  they  want 

Usable 

4 

The  application  must  be  intuitive,  clear,  concise, 
‘friendly’,  etc.  to  the  user  population 

Customisable  interface  (especially  if 
proprietary  application) 

4 

Appearance  of  the  application  should  be  customisable 
to  the  needs  and  whims  of  the  user 

First  impression  positive 

4 

Application  should  immediately  make  the  user  want  to 
use  it 

Supports  heterogeneous  platforms 
(including  PDAs)  and  Operating  Systems 

1 

In  general,  should  support  laptops,  desktops, 

Windows  OS,  Mac  OS,  Palm  OS,  Windows  CE, 
Blackberry,  Sun,  Unix,  Linux,  etc. 

Interoperable 

2 

All  applications  and  platforms  should  be  able  to  share 
all  data  with  no  data  transformation  needed 

Supports  heterogeneous  users 

3 

Should  support  the  needs  of  engineers,  scientists, 
administrators,  etc. 

Is  scalable  to  required  number  of  users 

3 

Can  be  used  by  as  few  or  as  many  users  as  required 
by  the  project 

Has  automatic  archiving 

2 

No  need  for  the  user  to  save  files  nor  to  record  what 
version  of  the  file  it  is 

Has  automatic  backup 

2 

No  need  to  backup  to  a  more  secure  site  files  that  are 
already  saved 

Has  a  clear,  modular,  flexible  structure 
which  mirrors  topic  area 

1 

Organisation  of  collaborative  site  should  mirror  users’ 
mental  models  of  the  project 

No  requirement  to  follow  a  linear  work 
path 

3 

Users  should  be  able  to  use  a  desired  function  at  any 
time,  and  not  be  held  to  perform  tasks  in  a  strict  order 
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Requirement 

Priority 

Definition 

Audit  Trail  available 

2 

The  trail  that  leads  to  the  current  file,  configuration, 
etc.  should  be  clear  for  auditing,  management 
purposes 

Permits  access  to  other  networks 

1 

Allows  access  to  DWAN,  building  networks,  WWW, 
etc.  even  though  it  may  be  its  own  discrete  network 

Maintains  Atomicity 

2 

Related  exchanges,  separated  by  time  should  keep 
their  relationship  (i.e.  not  obscure  their  relationship) 

Extensible 

4 

The  application  should  support  add  on  extensions  to 
its  capabilities,  functionality,  appearance,  etc. 

Registration/log  on/off  procedure 

4 

To  monitor  who  accessed  the  application  and  when, 
and  to  support  the  notion  of  personnel  presence 

Selective  Access  (control  of  who  sees 
what  when) 

4 

To  manage  issues  surrounding  security  classification 
and  to  avoid  a  team  member  setting  to  work  on 
incomplete  guidance  documents,  etc. 

Easy  to  Install 

4 

There  should  be  few  actions  required  of  the  installer 

Easy  to  Maintain 

4 

There  should  be  few  actions  required  to  keep  the 
application  running  well 

Priorities  were  given  a  reverse  order  so  that  a  high  priority  requirement  would  result  in  a  high 
score.  Thus  a  ‘5’  reflects  a  high  priority.  As  can  be  seen  from  Table  6  above,  there  are: 

•  18  requirements  at  priority  ‘5’  (highest) 

•  13  requirements  at  priority  ‘4’ 

•  7  requirements  at  priority  ‘3’ 

•  8  requirements  at  priority  ‘2’;  and 

•  9  requirements  at  priority  ‘  1  ’  (lowest) 

An  evaluation  matrix  (Figure  1)  was  used  to  compare  the  different  tools.  The  scores  for  uses/goals 
were  summed,  with  points  being  awarded  as  follows: 

•  Satisfied  -  1 

•  Partially  satisfied  -  0.5 

•  Not  satisfied  -  0 

•  Unknown  -  0 

An  overall  score  was  then  calculated  for  each  tool  equalling  the  sum  of  the  weighted  requirement 
scores  added  to  the  sum  of  the  scores  for  uses/goals. 
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5.3  Results 


The  evaluation  matrix  was  used  to  evaluate  five  collaborative  tools:  SharePoint,  NetMeeting, 
LiveLink,  Groove  and  Google  Docs  and  can  be  found  in  Annex  C.  The  analysis  was  based  on 
information  learned  about  the  different  tools  from  the  SMEs,  and  information  contained  on  the 
tools  respective  websites.  The  numbers  that  resulted  are  not  rounded  and  are  rather  large. 

Emphasis  should  be  placed  on  the  rank  and  general  clumping  of  values  with  little  emphasis  placed 
on  the  actual  values.  Out  of  a  possible  high  score  of  866  463,  the  overall  scores  for  each  tool  were: 

•  SharePoint  -  293  182.5 

•  Groove  -  277  104 

•  Google  Docs  -  120  664.3 

•  LiveLink  -  1 14  224 

•  NetMeeting  -  56  375 

SharePoint  ranked  overall  as  the  most  appropriate  collaborative  tool  based  on  this  scoring  system. 
However,  it  is  useful  to  consider  the  score  of  each  tool  for  each  of  the  uses/goals.  This  information 
is  presented  in  Table  7.  The  highest  score  is  shaded.  Note  that  the  Total  Possible  (second  from  the 
right  column)  for  each  of  the  Uses/Goals  are  not  the  same  as  they  vary  according  to  the  number  and 
priority  of  requirements  relevant  to  that  Use/Goal  (i.e.  number  of  shaded  boxes  in  Figure  1  varies). 

Table  7:  Tool  Scores  for  Each  Use/Goal 


Tool 

Share 

Point 

Net 

Meeting 

Live 

Link 

Groove 

Google 

Docs 

Total 

Possible 

Highest 

% 

Satisified 

Communication 

27 

20 

20 

24 

22.5 

38 

71% 

Design 

24.5 

15 

10 

24 

22 

38 

64% 

File  sharing 

22.5 

15 

17 

20 

19 

32 

70% 

File  storage 

13.5 

1 

10 

11 

14.5 

20 

73% 

Archiving 

14.5 

1 

11 

11 

12.5 

22 

66% 

Simulation/demonstration 

0.5 

1 

8 

10 

3 

28 

36% 

Testing/  validation 

0.5 

1 

13 

11 

19 

30 

63% 

Planning 

13 

8 

6 

9 

0 

17 

76% 

Project  monitoring/control 

20 

8 

9 

15 

0 

24 

83% 

Workflow  control 

15 

1 

5 

12 

0 

22 

68% 

Security/access  control 

15.5 

10.5 

8 

12 

15 

20 

78% 

Social  activities/  team  building 

18 

17.5 

14 

17 

18 

26 

69% 

Automated  project  support 

20 

1 

13 

18 

0 

29 

69% 

Resource  allocation 

16.5 

1 

7 

13 

0 

19 

87% 

Decision  support 

18.5 

1 

2 

13 

0 

25 

74% 

Analysis  support 

17.5 

1 

3 

12 

0 

25 

70% 

Team  coordination 

20 

1 

14 

18 

0 

27 

74% 

13 

12.5 

4 

16 

19 

35 

54% 

Network  development 

1 

8.5 

2 

10 

16 

30 

53% 
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The  far  right  column  (Highest  %  Satisified)  takes  the  highest  score  from  each  Use/Goal  (i.e.  the 
shaded  box)  and  divides  it  by  the  Total  Possible  value  to  give  an  idea  of  how  well  that  Use/Goal  is 
met  by  the  highest  scoring  tool.  Note  that  there  is  considerable  variance  on  the  level  of  satisfaction 
between  the  tools,  often  due  to  some  tools’  complete  lack  of  support  for  particular  uses/goals. 

As  can  be  seen  from  Table  7,  SharePoint  has  the  most  ‘high  scores’,  followed  by  Google  Docs  and 
then  Groove.  In  spite  of  this,  Groove  still  scored  higher  overall  than  Google  Docs,  indicating  that 
Google  Docs  is  ranked  better  for  certain  uses  (e.g.  File  Storage)  but  Groove  is  ranked  better 
overall. 

Although  the  maximum  score  for  each  use/goal  is  variable  (according  to  the  number  of 
requirements  that  are  relevant  to  that  use/goal),  this  information  can  still  be  used  to  identify  where 
research  should  be  focused.  For  instance,  ‘10’  was  the  highest  score  for  collaborative  tools  to 
support  ‘simulation/demonstration’.  The  maximum  score  it  could  have  achieved  is  ‘28’  assuming 
all  of  the  relevant  requirements  were  ‘Satisfied’  which  gives  the  value  in  the  far  right  column  of 
36%  (i.e.  10/28).  This  implies  two  things:  (1)  that  there  are  not  many  collaborative  tools  that 
support  this  use/goal;  and  (2)  that  of  those  tools  that  exist,  they  do  not  satisfy  the  requirements  for 
this  use/goal  very  well.  Thus  it  may  be  appropriate  to  say  that  money  and  effort  should  be  spent  to 
develop  collaborative  means  to  support  ‘simulation/demonstration’. 

5.4  How  to  Evaluate  Tools  using  the  Evaluation  Matrix 

The  use  of  the  evaluation  matrix  to  evaluate  tools  is  fairly  straight  forward.  In  the  first  instance, 
the  evaluator  should  write  a  brief  description  of  the  tool  in  cell  Bl2.  This  description  should  be 
enough  to  tell  a  reader  what  the  tool  is  intended  to  do,  and  where  to  go  for  further  information  (i.e. 
a  hyperlinked  URL). 

Assuming  the  evaluator  has  used  the  tool  or  has  enough  understanding  about  the  tool  (through 
documentation,  marketing  material),  s/he  should  then  proceed  to  consider  each  relevant 
requirement  for  each  use/goal.  Relevant  requirements  are  those  cells  that  are  not  shaded.  There 
will  be  more  relevant  requirements  for  some  uses/goals  than  for  others.  It  is  recommended  that  the 
evaluator  go  through  all  the  requirements  (e.g.  all  columns)  for  a  single  use/goal  before  moving  on 
to  the  next  use/goal  (e.g.  row)  as  illustrated  in  Figure  2  below. 

We  recognise  that,  in  practice,  however,  most  evaluators  are  going  to  fill  in  blank  cells  in  a  hybrid 
systematic/opportunistic  manner.  This  arises  from  the  fact  that  the  evaluator  knows  what  other 
uses/goals  will  be  evaluated  and  cannot  help  but  evaluate  more  than  one  when  considering  a 
requirement. 

For  each  blank  cell,  the  evaluator  must  determine  whether  the  requirement  is  satisfied  (Y),  not 
satisfied  (N),  partially  satisfied  (P),  or  unknown  (U)  as  illustrated  in  Figure  3.  For  the  purposes  of 
our  work,  if  an  application  did  not  support  a  use/goal,  we  entered  ‘N’  in  the  spreadsheet,  reflecting 
that  the  requirement  is  not  addressed  but,  more  importantly,  scoring  a  ‘0’  for  that  cell. 


2  References  to  specific  cells  will  be  made  with  respect  to  the  column  letter  and  the  row  number. 
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Figure  3:  Evaluation  Matrix  with  Partial  Entries 
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If  the  evaluator  enters  ‘P’  (for  ‘Partially’)  the  evaluator  should  note  down  why  it  is  a  partial  match 
at  the  bottom  of  the  spreadsheet  for  future  reference.  Due  to  time  constraints,  this  was  not  done 
during  this  project. 

The  spreadsheet  automatically  updates  the  scores  as  the  evaluator  fills  in  the  blank  cells.  Total 
scores  for  each  requirement  are  shown  at  the  bottom  of  each  requirement  (row  21),  and  totals  for 
each  use/goal  at  the  far  right  of  each  use/goal  (column  BF).  Total  score  is  found  in  the  lower  right 
hand  comer  (BF21).  Thus,  immediately  upon  finishing  the  evaluation,  the  evaluator  will  know 
how  suitable  the  collaborative  tool  is  for  the  purpose  in  mind. 

5.5  How  to  Select  Tools  for  Specific  Purposes 

Once  a  number  of  tools  have  been  evaluated  it  will  then  be  possible  to  select  specific  tools  for 
specific  uses/goals,  based  on  their  score.  This  is  not  meant  to  imply  that  the  tools  are  interoperable 
where  a  tool  ideal  for  some  use/goal  makes  available  relevant  information  to  another  tool  that  is 
best  for  another  use/goal.  Indeed,  it  is  likely  that  many  of  the  tools  listed  are  actively  antagonistic 
to  each  other  with  little  to  no  information  sharing  across  tools.  In  general,  each  of  the  different 
tools  are  completely  separate  systems. 

The  first  step  toward  selecting  the  most  appropriate  tool  based  on  the  evaluation  matrix  is  to 
determine  what  the  tool  is  meant  to  support.  This  may  be  a  single  use/goal,  but  is  more  likely  to  be 
some  combination  the  19  uses/goals.  Having  determined  what  uses/goals  the  tool  should  support, 
it  may  be  necessary  to  prioritise  between  them.  If  the  most  important  uses/goals  actually  scored 
low,  but  the  overall  sum  is  raised  by  high  scores  for  other  uses/goals,  the  then  the  highest  scoring 
tool  may  not  actually  be  the  best  one  for  that  job.  For  example,  referring  back  to  the  five  tools 
presented,  if  the  important  uses/goals  in  selecting  a  collaborative  tool  for  a  specific  project  were 
‘File  Storage’  and  ‘Social  Presence  and  Team  Building’  then  the  two  highest  ranking  tools 
(SharePoint  and  Groove)  appear  to  be  a  less  effective  choice  compared  to  the  top  ranking  tool  for 
those  specific  use/goals:  Google  Docs. 

Thus,  the  selection  of  a  tool  for  a  specific  purpose  is  made  by  considering  the  sum  of  scores  for  the 
uses/goals  of  interest,  and  perhaps  the  specific  score(s)  of  the  highest  priority  uses/goals. 

5.6  How  to  Identify  R&D  Requirements 

There  are  two  ways  that  research  and  development  requirements  can  be  identified:  (1)  by  looking  at 
requirement  scores;  and  (2)  by  looking  at  use/goal  scores. 

Requirement  scores  are  variable  according  to  the  number  of  uses  to  which  a  requirement  applies. 
However,  a  reader  can  get  a  quick  overview  from  the  evaluation  matrix  of  how  well  a  requirement 
is  not  met  purely  from  the  high  concentration  of  red  colour  coding  in  the  vertical  dimension.  Some 
requirements  have  been  particularly  poorly  met  by  all  tools  evaluated;  for  example,  the  provision  of 
a  searchable,  writable  database  with  reusable  classes. 

The  other  way  of  identifying  R&D  opportunities  is  by  considering  the  scores  for  each  use/goal. 
Again,  these  scores  are  variable  according  the  number  of  times  a  use/goal  was  assessed  against  a 
requirement.  Similarly  to  the  requirements  though,  the  reader  can  get  a  quick  overview  of  the  need 
for  R&D  for  uses/goals  by  looking  for  blocks  of  red  in  the  horizontal  dimension.  This  represents  a 
use/goal  not  meeting  the  corresponding  requirements.  Again,  an  example  of  two  candidates  for 
further  R&D  are  ‘simulation/demonstration’  and  ‘network  development’. 
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This  tool  really  only  provides  the  reader  with  information  to  help  them  identify  R&D  requirements. 
It  is  not  a  decision  making  tool,  and  cannot  substitute  for  the  decision  maker’s  own  understanding 
of  the  application  domain  and  the  options  that  exist  therein. 

5.7  Suggestions  to  improve  the  Evaluation  Matrix 

The  evaluation  matrix  is  undoubtedly  daunting  to  use;  however,  an  iterative  process  has  reduced  a 
bigger  list  of  requirements  and  uses/goals  to  the  current  set.  As  noted  elsewhere,  there  are  19 
uses/goals  and  55  requirements,  leading  to  the  possibility  of  filling  in  1045  discrete  cells.  These 
discrete  cells  have  been  further  limited  by  shading  cells  that  were  deemed  to  be  inapplicable; 
however,  there  are  still  many  judgements  to  be  made  and  an  ‘auto-fill’  function  is  not  appropriate 
as  there  are  no  multiple  cell  sets  that  will  necessarily  receive  the  same  entry.  The  authors  do  not 
believe  it  is  possible  to  reduce  the  number  of  judgements  that  need  to  be  made  at  this  time. 

It  is  noted,  however,  that:  (1)  the  scores  that  result  cannot  be  directly  compared  with  each  other; 
and  (2)  the  resulting  evaluation  scores  are  rather  large  and  cumbersome.  It  is  advisable  that  each 
score  should  be  divided  by  the  number  cells  in  that  row  or  column  in  order  to  standardise  the  score. 
Differences  won’t  be  so  clear,  but  direct  comparisons  will  be  possible.  This  normalization  process 
was  not  done  during  this  project  due  to  time  constraints. 

Ideally,  it  would  be  easier  to  enter  information  about  ‘partial’  scores.  At  the  moment  the 
spreadsheet  makes  not  provision  to  record  or  store  these,  without  affecting  the  appearance  of  the 
evaluation.  The  use  of  ‘comments’  in  the  cell  itself  is  one  solution. 
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6  Conclusions  and  Recommendations 


The  work  reported  here  identifies  a  number  of  tools  suitable  for  collaborative  planning  and 
engineering.  Further,  we  have  identified  a  number  of  user  requirements  for  such  tools  that  would 
help  to  ensure  that  investment  in  such  tools  would  be  beneficial.  We  have  also  identified  the  most 
likely  uses  for  collaborative  planning  and  engineering  tools.  This  work  led  to  the  development  of 
an  evaluation  matrix  for  use  when  assessing  such  tools.  This  evaluation  matrix  can  also  be  used  to 
select  which  tool  to  use  and  to  identify  R&D  requirements,  thus  leading  to  more  strategic  use  of 
available  funds  to  address  the  most  significant  problems. 

The  evaluation  matrix  has  been  used  to  evaluate  a  number  of  collaborative  tools  currently  used  by 
DND.  Of  these,  the  highest  overall  score  was  SharePoint,  followed  by  Groove.  These  two  tools 
present  a  single  workspace,  from  which  users  can  ‘check  out’  documents  to  be  worked  on. 
Flowever,  they  do  not  provide  real-time  collaborative  working  with  awareness  of  the  moment-by¬ 
moment  actions  of  the  collaborators. 

It  is  not  recommended  that  all  the  tools  identified  should  be  evaluated.  However,  it  is 
recommended  that  any  tool  that  is  proposed  for  use  by  DND  could  be  evaluated  as  a  first  step  to 
determine  whether  it  provides  any  significant  advantage  over  the  existing  capability. 

It  is  also  felt  that  the  tool  list  provided  should  be  considered  for  which  tools  may  represent 
improvements  on  those  already  used  by  DND.  These  tools  should  be  subject  to  an  evaluation  to 
establish  their  relative  merit  amongst  the  tools  evaluated  for  this  project. 

Pursuant  to  this,  the  evaluation  matrix  should  be  improved  such  that  it  is  better  able  to 
accommodate  comments  about  specific  judgements. 

The  scoring  system  incorporated  in  the  evaluation  matrix  should  also  be  amended  to  permit  direct 
comparisons  between  different  scores.  This  would  help  tool  selection  and  identification  of  R&D 
needs. 

Further  to  Canadian  efforts,  there  has  been  a  large  research  project  in  Europe  investigating  the 
FEDEP  (Brasse,  Mevassvik,  &  Skoglund,  2000)  and  this  has  resulted  in  evaluations  of  a  number  of 
tools.  A  similar  effort  has  been  undertaken  in  the  United  States  (Cowen,  2007),  although  it  focuses 
on  collaborative  tools  for  crisis  management.  Although  useful  to  inform  the  Canadian  effort, 
neither  of  these  has  resulted  in  outcomes  that  can  be  easily  assumed  by  the  Canadian  simulation 
community,  largely  because  the  particular  application  domain.  None  of  the  tools  investigated  by 
the  European  and  US  efforts  have  been  evaluated  for  this  project  as  the  authors  decided  to  only 
evaluate  tools  that  were  mentioned  in  the  SME  interviews. 

As  a  result,  we  feel  that  this  project  has  provided  an  important  first  step  toward  the  technological 
support  of  distributed  planning  and  engineering  teams.  The  evaluation  matrix  can  be  used  to 
strategically  allocate  effort  and,  ultimately,  can  play  an  important  role  in  developing  a  tool  that  can 
lead  to  more  effective  staging  of  distributed  simulations. 
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Title 

Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

Distributed 
collaboration  for 
engineering  and 
scientific 
applications 
implemented  in 
Habanera, 
a  Java-based 
environment 

Brent  Driggers, 

Jay  Alameda, 

Ken  Bishop 

Concurrency: 

Practice  and 
Experience, 

Vol  9  (11), 
pp  1269-1277, 
(November  1997) 

Communication  (text/chat  are  good  for 
documenting  and  archiving,  but  bad  for 
communications  -  audio  better),  voting 
(to  establish  operating  parameters), 
reactor  design,  wind  tunnel  simulation 

Display  same  material  on  every  computer,  synchronisation, 
determine  state  of  remote  computer,  create  session  or  join 
session,  become  session  manager 

Habanera,  KESI 

Madefast: 
Collaborative 
engineering  over 
the  Internet 

Mark  R. 

Cutkosky, 

Jay  M, 

Tenenbaum, Jay 
Glicksman 

Communications  of 
the  ACM,  September 
1996,  39  (9). 

Repository  of  info 

Chronology  of  meetings  and  milestones,  map  of  registered 
participants,  top  level  pages  for  design  process  and  design 
artefacts,  computer  assisted  design  (CAD)  models,  notes,  test 
results,  calculations,  other  design  information,  authoring, 
document  control,  document  navigation,  asynchronous 
communication,  synchronous  communication,  support 
heterogenous  platforms,  scale  well  in  WANs,  enable  capture  of 
session  info,  facilitate  playback  of  asynchronous  infonavigation, 
organisation  of  data/info 

MADE  (Manufacturing  Automation 
and  Design  Engineering) 

Java's  role  in 

distributed 

collaboration 

Marina  Chen, 
James  Cowie 

Concurrency: 

Practice  and 
Experience, 

Vol  9(6), 
pp  509-519 
(June  1997) 

Proof  of  concept  demonstrations, 
testing/validation,  planning  (process 
design,  partition  of  collaboration  into 
tasks,  specification  of  public  interfaces, 
binding  of  interfaces  to  a  publication 
'home') 

Flexibility,  cost  constraining,  support  for  large-scale 
collaborations,  raw  performance,  allows  quick  POC 
demos/validations,  publish  code  and  documentation  in  standard 
format,  configuration  control,  collaboration  design,  shared  name- 
space  management,  flow  control  for  links  between  components, 
security,  resource  management,  software  integration 

Java,  C++,  Tcl/Tk,  Perl  5,  Standard 

ML,  F77,  MPI,  F95,  HPF,  HTML 

Media 

spaces:  bringing 
people  together  in  a 
video,  audio,  and 
computing 
environment 

Sara  A.  Bly, 

Steve  R. 

Harrison,  Susan 
Irwin 

Communications  of 
the  ACM,  January, 
1993, 

Vol  36,  No  1 

Awareness,  chance  encounters,  locating 
colleagues,  video  phone  conversations 
(focused,  group  discussions, 
recording/replaying  video  records, 
project  support  (experimentation  and 
envisionment),  presentations,  social 
activities 

Scalability,  points  of  reference  (spatial,  object,  figural  [people]) 

Audio,  video,  computing, 

VideoWindow:  Bellcore;  Cruiser: 
Bellcore:  RAVE:  Rank  Xerox 
EuroPARC; 

CAVECAT/Telepresence:UofT ; 
TeleCollaboration:  US  West 

Advanced  Technologies;  Kasmer: 

Xerox  PARC 

DistView:  support 
for  building  efficient 
collaborative 
applications  using 
replicated  objects 

Atul  Prakash, 
Hyong  Sop  Shim 

Computer  supported 
collaborative  work: 
proceedings  of  the 

1994  ACM 
conference  on 
computer  supported 
collaborative  work, 
Chapel  Hill,  NC,  pp. 
153-164 

Export  window  to  group,  import  window, 
communication,  shared  editing, 

Ability  to  share  synchronised  views  of  interactions  with  an 
application,  objective-level  replication  scheme,  efficient,  fault- 
tolerant,  allow  private  application  windows  in  collaborative 
environment,  simultaneous  interaction  with  application,  no  drop  in 
system  response  times,  equal  and  adequate  performance  across 
WAN,  minimal  additional  effort  to  turn  non-coilaborative  app  into 
collaborative  one,  no  additional  knowledge  required  on  the  part  of 
the  user  to  use  collaborative  app,  feedback,  chat,  identification, 
synchronisation  of  actions  in  shared  windows, 
autoupdate/synchronisation,  lock  object, 

DistView 
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Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

Scenarios  as  a  Tool 
for  Collaborative 
Envisioning:  Using 
the  Case  of  New 
Sensor 

Technologies  for 
Military  Urban 
Operations 

Joshua 
Schoenwald, 
Stoney  Trent, 
James  Tittle, 

David  Woods 

Proceedings  of  the 
Human  Factors  and 
Ergonomics  49th 
Annual  Meeting, 

2005. 

n/a 

Organises  large  amounts  of  information,  navigation  should  be  a 
model  of  the  topic  being  navigated,  multi-tiered  organisation  (3-5 
meaningful  sections),  no  required  linear  path 

Topic  Landscape  -  not  really  relevant 
to  engineering,  but  interesting  for 
research 

Team  ergonomics 
and  human 
engineering 
methods  for  the 
design  of 
collaborative  work 
environments:  a 
case  study 

Bohan  J  A, 

Wallace  D  F 

Proceedings  of  the 
Human  Factors  and 
Ergonomics  41st 
annual  meeting, 

1997. 

Picture  building  in  ops  centres 

Enables  direct  communication,  supports  tasks  to  be  performed, 
enables  efficient  and  accurate  acquisition  of  required  information 

n/a 

Collaborative 

Logistics: 

Developing  a 
Framework  to 

Evaluate  Socio- 
Technical  Issues  in 
Logistic-Based 
Networks 

Joseph  Lyons, 

Jill  Ritter,  Krystal 
Thomas,  Laura 
Militello,  Patrick 
Vincent 

Collaborative 
Technologies  and 
Systems,  2006.  CTS 
2006.  International 
Symposium  on.  May 
2006. 

pp  208-  214 

Distributed  adaptive  logistics/sense  and 
respond  logistics/knowledge  enabled 
logistics 

Speed,  robustness,  evaluation  metrics,  address  socio-technical 
factors,  compensate  for  loss  of  socio-emotional  cues,  facilitate 
trust-building  between  team  members,  foster  team  cohesion, 
avoid  local  coalitions,  positive  leadership 

E-mail,  phone,  video  conference, 
instant  messaging 

Internet-Based 
Collaborative  Virtual 
Simulations  with 

Area  of  Interest 
Management 

A.  Diabi, 

S.  Shirmo- 
hammadi, 

A.  Gillmore, 

P.  Lacombe, 

J.  C.  de  Oliveira 

Collaborative 
Technologies  and 
Systems,  2006.  CTS 
2006.  International 
Symposium  on.  May 
2006. 

pp  200-  207 

Mainly  for  use  during  simulations,  not  in 
the  lead  up  to  a  simulations  -  mostly 
about  overcoming  bandwidth/latency 
restrictions 

End  to  end  delay  of  less  than  100  (or  200)  ms,  scalability, 
accurate  and  regular  state  information 

IP  Multicast,  Application  Layer 
Multicasting  (ALM),  Area  of  Interest 
Management  (AolM) 

Awareness-Enabled 
Coordination  for 

Large  Scale 

Collaboration 

Management 

Dimitrios 
Georgako- 
poulos,  Marian 
Nodine,  Donald 
Baker,  Andrzej 
Cichocki, 

Collaborative 
Technologies  and 
Systems,  2006.  CTS 
2006.  International 
Symposium  on.  May 
2006. 

pp  132- 141 

Homeland  security,  intelligence 
gathering 

Support  many  users,  support  single  users,  support  teams, 
increase  efficiency,  context  management,  contextualisation,  team 
coordination,  process  automation,  policy  enforcement,  situational 
awareness,  team  awareness,  permit  dynamic  change/adaptation, 
reduces  environment  presented  to  user  (context  mgmt) 

Awareness-enabled  coordination 

Specifying  Agent 
Support  for 
Collaboration 

Igor  Hawry- 
szkiewycz, 

Proceedings  of  the 
International 
Symposium  on 
Collaborative 
Technologies  and 
Systems.  2006. 
pp  109  - 116 

Agents  to  encourage  technologically 
mediated  collaborative  support 

Contextual  dynamic  facilitation  by  software  agents,  agents  that 
'read1  files  when  they  change  and  determine  the  impact  and 
change  other  things  accordingly,  accommodates  activities  and 
work  items,  work  processes  (including  events  and  workflows), 
and  social  structures,  different  agents  (Activity,  Work-Action, 

Role,  Group,  Personal,  Artifact,  Connect  (Broker),  Coordination), 
support  perception,  reasoning  and  action 

E-mail,  SMS,  visual  displays,  web 
portals,  blogs,  file  systems, 
discussion  systems,  workflow 
systems,  calendar  systems,  group 
calendars,  shared  whiteboards, 
workspaces  for  asynchronous 
support,  interactive  visual  displays 
and  workspaces 
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Title 

Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

VO-based  Dynamic 
Security 

Associations  in 
Collaborative  Grid 
Environment 

Yuri 

Demchenko, 

Cees  de  Laat, 
Vincenzo 

Ciaschini, 

Valerio  Venturi 

Proceedings  of  the 
International 
Symposium  on 
Collaborative 
Technologies  and 
Systems.  2006. 
pp  38-47 

managing  dynamic  security  associations 
and  complex  resource  provisioning 

dynamic  trust  management,  span  multiple  trust  domains,  handle 
different  user  identities  and  attributes,  etc.,  attributes  and 
metadata  resolution  and  mapping,  policy  combination  and 
aggregation,  flexible  management  infrastructure,  registration 
procedure,  security  policy,  acceptable  use  policy 

n/a 

A  Longitudinal 

Study  of  the  Use  of 
a  Collaboration 

Tool:  A  Breadth  and 
Depth  Analysis 

Jean  Scholtz, 

Emile  Morse, 
Michelle  Potts 
Steves 

Proceedings  of  the 
International 
Symposium  on 
Collaborative 
Technologies  and 
Systems.  2006. 
pp  1-11 

share  files,  manage  projects,  create 
solutions,  chat, 

No  additional  overhead  associated  with  using  the  system, 
awareness  of  who  is  online,  who  is  a  member,  network 
management  issues,  should  assist  (not  hinder)  decision  making, 
synchronous  and  asynchronous  working,  text  chat,  audio  chat, 
net  meeting,  designation  of  'new'  or  'changed',  access  role,  value- 
added,  overheads  to  utilisation,  impacts  on  individuals,  metrics 
(did  adhoc  collaboration  increase,  did  tool  support  collaboration, 
how  did  work  process  change,  how  did  work  products  change), 
no  time  to  synchronise  space 

Groove  (file  sharing,  basic 
collaboration,  calendar,  sketch  pad, 
note  pad,  forms  tools  project 
planning  tool,  meeting  space  support 
tool,  collaborative  browser  -  all 
compatible  with  MS  Office  tool  suite) 

Collaboration 

Entities  on 
Deterministic  Finite 
Automata 

Minjun  Wang, 
Geoffrey  Fox, 
Marlon  Pierce 

Proceedings  of  the 
International 
Symposium  on 
Collaborative 
Technologies  and 
Systems.  2006. 
pp  26-37 

Shared  presentation  editing 

Master/client  active/passive  relationship,  synchronised  views 

Powerpoint,  Impress,  ReviewPlus 

WORKPAD:  an 
Adaptive  Peer-to- 
Peer  Software 
Infrastructure  for 
Supporting 
Collaborative  Work 
of  Human  Operators 
in 

Emergency/Disaster 

Scenarios 

Massimo 

Mecella,  Michele 
Angelaccio, 

Alenka  Krek, 
Tiziana  Catarci 

Collaborative 
Technologies  and 
Systems,  2006.  CTS 
2006.  International 
Symposium  on.  May 
2006. 

pp  173- 180 

Disaster  response,  short  term  recovery 
(software  and  communication) 

Peer-to-peer  architecture,  novel  techniques  for  knowledge 
content  integration,  novel  adaptive  techniques  for  cooperative 
work  and  workflow  management,  geo-referenced  information, 
devising  safety  and  security  solutions  for  emergency 
communications,  threat  detection  and  management,  wireless 
communications  and  back  end  networks,  geo-collaboration 

WORKPAD,  content  management 
systems  (CMS)  middleware  (but 
doesn't  say  what) 

Cooperative 
environments  for 
distributed:  the 
distributed  systems 
environment  report 

B.  Baurens,  P. 
Chilaev,  V. 

Krivtsov,  V. 
Volochinov 

Springer  Lecture 

Notes  In  Computer 
Science  Series , 
Cooperative 
environments  for 
distributed:  the 
distributed  systems 
environment  report. 
2002. 
pp  7  - 13 

n/a 

n/a 

n/a 
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Title 

Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

Collaborative 

Engineering 

Enterprise 

Suleyman 

Guleyupoglu 

Extensible  Modeling 
and  Simulation 
Framework  (XMSF) 
Technical 

Opportunities 

Workshop. 

httos://www.movesin 

stitute.ora/xmsf/work 

shoD/aulevuDoalu/aul 

evuDoaluXmsfPointP 

aoer.odf. 

Simulation-Based  Acquisition, 
information  sharing  and  decision  support 
by  discussing,  analysing,  and  iterating  all 
aspects  of  system  design  by  using 
shared  information,  models  and 
applications 

Scalable  client-server  architecture,  multiple  platforms  (including 
PDAs),  asynchronous  availability  of  information,  supports  project 
management  by  tracking  and  monitoring  tasks,  provides  tools  to 
help  define  tasks  in  collaboration,  create  virtual  enterprise  via 
persistent  virtual  space,  support  virtual  personal  office,  provide 
framework  for  intergrating  simulations  and  diverse  collaborative 
activities,  enhance  productivity,  generate  products,  faster, 
automation,  integration,  multiple  individuals  working  on  same 
work  process  flow  concurrently,  low  bandwidth  VOIP,  commercial 
off  the  shelf  (COTS)  collaboration  tool  integration  (e.g.  remote 
conferencing),  configuration  control  implantation  for  the  repository 

n/a 

Communication 
Framework  for 

XMSF 

Dr.  Norbert 
Schiffner 

httDs://www.movesin 

stitute.ora/xmsf/work 

shoD/schiffner/Comm 

unicationFrameworkF 

orXMSF.odf 

Share  media  across  a  network  among 
two  or  more  workstations,  more  than  one 
person  at  a  time  can  generate,  process, 
represent  and  store  info 

Consistency  of  distributed  data  (maintenance  of  atomicity,  first-in 
first-out  ordering,  causality  preservation),  efficient  session 
management  (floor  control,  session  control),  network  as  a  'black 
box',  heterogeneity  of  users,  heterogeneity  of  hardware, 
heterogeneity  of  software,  heterogeneity  of  communication, 
interoperability,  flexibility,  extensibility,  scalability  (one  to  one,  one 
to  many,  many  to  many),  Ergonomy,  usability,  customizability  of 
interface,  modular  architecture,  reusable  classes, 
modules/libraries,  executable  components,  frameworks,  reliable, 
robust,  efficient 

HOUCOM 

Federation 

Composition 

Process  and  Tool 
Support  in  EUCLID 
RTP  11.13 

Marco  Brasse; 

Ole  Martin 
Mevassvik;  Tom 
Skoglund 

TNO  (Netherlands) 
and  FFI  (Norwegian 
Defence 

Organization),  2000. 

Design  federation,  implement  federation, 
selection  of  simulation  assets  and 
composition  of  these  assets  into  a 
working  HLA  federation,  control  work 
flow  by  guiding  user  through  SEDEP 
process  step  by  step:  1.  federation 
system  spec  is  processed  and  presented 
to  the  user;  2.  User  develops  federation 
architecture  (iterative  -  decide  on  number 
and  type  (simple/complex)  of  federates); 

3.  User  details  required  federate 
capabilities  (maintains  links  between 
design  and  federate  capabilities);  4.  User 
formulates  repository  queries;  5.  User 
selects  simulation  assets;  6.  User 
supported  by  FCT  in  assessing 
candidate  simulation  assets;  7.  User 
documents  design  decisions 

Process-oriented,  tool-oriented,  searching  asset  repository,  asset 
characterisation,  federation  conceptual  model,  scenario, 
requirements  (scenario  and  assets),  asset  reuse,  distributed 
asset  repository,  asset  characterisation  method,  component 
based  architecture  design,  SEDEP-wide  integrated  tool  set, 
Federation  system  specification  (federation  scenario,  federation 
conceptual  model,  federation  system  requirements),  federation 
design  specification,  identification  of  federation  bridges,  selection 
of  federates,  FOM  (interface  contract  between  federates), 
federation  agreement  (documenting  inter-operability  decisions  not 
covered  by  the  FOM)  -  technical  agreements  (federation 
initialisation  mechanism,  management  procedure, 
database/coordinate  system  specs,  data  collection  agreements, 
algorithms  for  'fair  fight'  conditions)  resource  agreements 
(availability  of  resources,  sharing  of  (restricted)  data)  project 
specific  agreements  (commercial  licenses,  demonstration 
agreements),  complex  federates  (composed  of  functional  building 
blocks  -  customisable,  reusable,  secure,  high  performance), 
agreement  matrix,  supports  top-down  and  bottom-up  federation 
composition 

Federation  Composition  Tool  (FCT) 
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Title 

Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

Recommen-  dations 
for  Conducting 
Real-time  Human- 
in-the-Loop 
Simulations  over  the 
Internet 

Thomas  Z. 
Strybell, 

Riva  Canton, 
Vernol  Battiste, 
Walter  Johnson, 
Kim-Phuonq  L. 
Vul 

Proceedings  of  the 
Human  Factors  and 
Ergonomics  Society 
50th  Annual  Meeting, 
San  Francisco,  CA. 
2006. 

Procedural  recommendations:  1. 

Establish  an  audio  and  visual 
communication  link  for  developers, 
experimenters  and  test  pilots  that  is  fast 
and  reliable;  2.  Create  and  maintain  a 
digital  repository  of  simulation  software, 
paper  forms  and  data  files  used  in  the 
simulation  (i.e.  an  FTP  or  Website, 
responsibility  for  maintenance  clearly 
identified,  this  ensures  that  all  sites 
configured  with  latest  software  versions, 
possibly  a  central  hard  drive  image 
stored  here);  3.  Establish  a  formal 
readiness  procedure  that  will  determine 
when  the  simulation  is  ready  for  test 
participants  (would  also  determine  when 
to  freeze  simulation  so  that  pilot  and 
experimenter  training  could  commence) 

Simulation  Experimental  Design  Recommendations:  Control  and 
assess  differences  in  performance  between  sites  -  don't  want  to 
confound  simulation  location  with  participant  performance.  In 
experiment,  they  tested  pilots  at  one  location  then  did  experiment 
at  another  site.  1  pilot  performed  worse  than  other  3,  but  can't 
say  cause.  Design  Suggestions:  Talk/View  others,  Digital 
repository  of:  software,  paper  forms,  data  files;  Straightforward 
approach  to  knowing  when  simulation  is  ready  for  test, 

AudioA/ideo  link,  FTP  site  or  Website,  maintainer  explicitly 
identified,  Formal  readiness  procedure 

n/a 

Argo:  a  system  for 

distributed 

collaboration 

H.  Gajewska;  J. 
Kistler; 

M.  Manasse;  D. 
Redell 

ACM  Multimedia. 

2004. 

Know  who  is  doing  what,  mobility 

Video,  Application  sharing,  Tele-pointing,  telepainting  (different 
colour  for  each  user),  teleportation  (can  log  in  from  any  location 
and  get  same  system  look/feel) 

Argo,  DiCE,  ShowMe,  ABC  (all  very 
old)  Shared  X  (proxy  server),  Trestle 
(supports  window  replication)  and 
Mbone  Applications  (e.g.  wb  -  a 
groupware).  Other  tools  being 
developed  are  Sun's  ShowMe,  DiCE, 
ABC. 

Distributed 
Collaborative 
Environments  for 

21st  Century 

Modeling  & 

Simulation 

William  K. 

McQuay 

htto://www.modelinaa 

ndsimulation. ora/text/ 

McOuav.html 

Simultaneously  view/discuss/edit  docs, 
allow  uses  to  track  progress  of 
tasks/documents  (versioning) 

Access  to  docs,  maintain  docs,  auto  notification  of  updates, 
synchronous  connectability  to  docs,  monitor  progress  of  tasks, 
versioning 

n/a 

Collaboration  with 
Ecological  Interface 
Design 

Catherine  Burns, 
Angela  Garabet 

Proceedings  of  the 

48th  Annual  Meeting 
of  the  Human  Factors 
and  Ergonomics 
Society,  543-546. 

2004. 

Used  Microsoft  NetMeeting  3.0 

n/a 

NetMeeting  3.0 
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Title 

Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

User-Centered 
Evaluation  of 
Multinational 
Communication  and 
Collaborative 
Technologies  in  a 
Network-Centric  Air 
Battle  Management 
Environment 

Robert  Bolia, 

Anna 

Langhorne, 

W.  Todd  Nelson, 
Michael  Vidulich 

Proceedings  of  the 
Human  Factors  and 
Ergonomics  Society 
48th  Annual  Meeting 
(pp.  731-735).  Santa 
Monica,  CA: 

Human  Factors  and 
Ergonomics  Society. 

Top  4  rated  technologies:  Data  Capture 
and  Replay,  Chat  and  messaging,  Data 
Visualization  Tools,  File  and  Application 
Transfer.  Bottom  4  rated:  Opinion 

Polling,  Automated  Decision  Support, 

Large  Shared  Displays,  and  Virtual 
Whiteboard.  Neither  top  nor  bottom: 
Interactive  Intelligent  Agents,  Video 
Teleconferencing  (VTC),  Broadcasts  and 
Alerts,  Expertise/Knowledge  locations, 
Automated  Workflow.  Rated  by  USAF, 
USN  and  RAAF  in  terms  of  Air  battle 
management  work  domain,  through 
filling  out  Collaborative  Interface 
Technology  Survey  developed  for  the 
study. 

Data  Capture  and  Replay,  Chat  and  messaging,  Data 

Visualization  Tools,  File  and  Application  Transfer  Bottom  4  rated: 
Opinion  Polling,  Automated  Decision  Support,  Large  Shared 
Displays,  and  Virtual  Whiteboard.  Neither  top  nor  bottom: 

Interactive  Intelligent  Agents,  Video  Teleconferencing  (VTC), 
Broadcasts  and  Alerts,  Expertise/Knowledge  locations, 

Automated  Workflow. 

Top  4  req'ts:  ways  to  capture  data, 
Realtime  text  exchange,  shared 
visuals,  file  sharing 

Collaborative  Tools 
and  Shared  Mental 
Models 

Cheryl  Bolstad, 
Cleotilde 

Gonzalez, 

John  Graham, 

Mike  Schneider 

Proceeding  of  the 
Human  Factors  & 
Ergonomics  Society, 
48th  Annual  Meeting, 
New  Orleans,  LA. 

2004. 

Face-to-face,  Video  Conferencing,  Audio 
Conferencing,  telephone,  networked 
radio,  chat/IM,  white  board,  file  transfer, 
program  sharing,  email,  groupware, 
bulletin  board,  domain  specific  tools 
Collaborative  processes:  planning, 
scheduling,  tracking  information, 
brainstorming,  document  creation,  data 
gathering,  data  distribution,  shared 
situational  awareness  (SA) 

Time  of  collaboration  (synchronous  or  asynchronous), 
predictability  of  collaboration,  place  of  collaboration  (co-loc  or 
dist),  degree  of  interaction.  Also,  characteristics  of  tools: 
recordable/traceable  (necessary  for  good  SA),  identifiable  (if 
individuals  using  tools  are  easily  identified  -  good  for  team 
formation  and  shared  SA),  structured  (specific  communications 
only  vs.  variety  of  info  types  allowed.  Info  types:  verbal,  text, 
spatial/graphical,  emotional  (fatigue,  workload,  competence, 
anxiety,  etc),  photographic,  video 

n/a 

A  Field  Study  of 
Collaborative  Work 
in  Network 
Management: 
Implications  for 
Interface  Design 

Rene  Chow,  Kim 

J.  Vicente 

Proceeding  of  the 
Human  Factors  & 
Ergonomics  Society 
pp  356-360,  2001, 

Database  of  service  requests;  voice 
communication 

Need  eyes/hands  for  investigating  issues  at  remote  locations, 
change  managers  (map  out  plans  to  add/remove/replace 
hardware) 

Phone  (coordinated  with  voicemail, 
pagers)  and  Email 

Collaboration 

Effects  on 

Distributed  Student 
Team  Performance 

M.E.  Reichert, 

A.L.  Williams, 

C.M.  Harvey 

Proceeding  of  the 
Human  Factors  & 
Ergonomics  Society. 
2001,  Vol  1,  pp  763- 
767 

5  variables  looked  at:  Team  Organization 
(how  teams  distributed  work  and  set 
deadlines),  correlated  with  grade  Team 
Co-ordination  (how  teams  adapted  and 
worked  with  one  another),  was 
fundamental  to  team  performance. 

Team  Interaction,  Team  Leadership,  and 
Team  Cohesion  were  not  discussed 
(hence  assumed  no  correlation). 

n/a 

n/a 
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Title 

Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

PRESTO: 
multimedia 
distributed  network 
for  collaborative 
work  in  command 
and  control 

J.  A.  Modrick 

Proceeding  of  the 
Human  Factors  & 
Ergonomics  Society, 
40th  Annual  Meeting. 
1996. 

Captures  essence/dynamics  of 
collaboration  session;  maintain  cohesion 
within  group 

Define  and  manage  shared  context 

n/a 

Building  Scalable 
and  high  efficient 

Java  Multimedia 
Collaboration 

Wenjun  Wu,  Tao 
Huang,  Geoffrey 
Fox 

Proceedings  of  the 
International 
Symposium  on 
Collaborative 
Technologies  and 
Systems.  2006.  pp 
18-25 

n/a 

n/a 

Global-MMCS  is  main  tool  in  article. 
Other  tools  mentioned  are:  Java 
Collaborative  Environment  (JCE, 
developed  by  NIST);  Java  Enabled 
Tele-collaboration  (JETS,  from  Univ. 
of  Ottawa).  Apple's  QuickTime  is 
primary  technology  used  to  support 
video. 

Collaboration  and 
Community  Grids 

Geoffrey  Fox 

Proceedings  of  the 
International 
Symposium  on 
Collaborative 
Technologies  and 
Systems/  2006.  pp 
419-428 

See/hear  others;  editable  exchanges  of 
text;  audio/video  exchange;  bulletin 
boards,  list  serves,  Wikis 

Synchronous  and  asynchronous  communications; 

Skype,  Wiki  pages. 

Federation 
Development  and 
Execution  Process 
(FEDEP)  Tools  in 
Support  of  NATO 
Modelling  & 

Simulation 

Turrel, 

Christopher; 

Brown,  Rick; 

Igarza,  Jean- 
Louis;  Pixius, 

Kay;  Renda, 
Fernando 

NATO  Research  and 
Technology 
Organization  Neuilly- 
sur-Seine,  France. 
2004. 

List  of  NATO  Fedep  Tools. 

n/a 

A  long  list  of  tools  were  in  this  report 
and  can  be  found  in  the  list  of  tools  in 
Annex  B. 

The  Impact  of 
Microsoft’s  Unified 
Communications 
Launch 

Zeus  Kerravala 

Enterprise 

Computing  and 
Networking,  August 
2006. 

Desktop  video,  pager,  email,  mobile 
phone,  collaborative  software,  web 
conferencing,  fax,  voicemail,  audio 
conferencing,  laptop,  phone,  room-based 
video,  messenging  software,  PDA 

IP  network,  IP  telephoneA/olP,  Desktop  Software,  Mobility  (ability 
to  replicate  user  experience  no  matter  where  they  are),  Security, 
Presence  Information  (ability  for  users  to  understand  another 
user's  availability  and  willingness  to  communicate) 

n/a 

A  Review  of  Team 
Collaboration  Tools 
Used  In  the  Military 
and  Government 

Seymour  Cowen 

Office  of  Naval 
Research  and 
SPAWAR.  2007. 

Blogs,  bluetooth,  sms/mms  (mentioned, 
though  not  discussed),  Chat,  IM,  pocket 
casting,  podcasting,  RFID,  RSS  (Really 
Simply  Syndication...  customizable 
pushed  data),  VTC,  VoiceXML,  VoIP, 
Webcasts,  Wikis 

n/a 

Citadel,  FlashMeeting,  Glance, 
Holocene,  Conversation  Mode, 
GoToMeeting,  Hexagon,  JotSpot, 
MERBoard,  Ourmedia, 
phpGroupWare,  R-CAST, 
smartMeeting,  Socialtext,  WiredRed 
Web,  Collaboration  at  Sea  (CAS), 
Collaborative  Information 

Environment  (CIE),  Defense 
Collaboration  Tools  Suite  (DCTS), 
InfoWorkSpace  (IWS),  Intelink, 

WebEx.  Plus  31  others. 
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Title 

Author(s) 

Source 

Uses/Goals 

Requirements 

Tools 

A  Framework  for 
Building 

Collaboration  Tools 
by  Leveraging 
Industrial 

Components 

Du  Li, 

Yi  Yang,  James 
Creel,  Blake 
Dworaczyk 

Coopls:  OnTheMove 
Federated 

Conferences  and 
Workshops.  2006. 

Group  text  editors,  group  sketch,  group 
calendar,  group  browser  and  group  to-do 
list. 

Tool  is  customizable  but  there  is  consistent  look  and  feel  across 
different  tools 

Evolvable  and  extensible 

Environment  for  Collaboration 
(EXEC) 

Vector  Approach  for 
Analyzing  Survey 
Questions 

Sophie 

Villeneuve, 

Phillip  S.E. 

Farrell 

Defence  Research 
and  Development 
Canada -Toronto, 

TM  2005-151.  2005. 

n/a 

n/a 

CIE  Portal,  Document  Manager,  Info 
Work  Space  (IWS),  Operations  Net 
Assessment  (ONS)  database,  Effects 
Based  Planning  Tools,  WebCOP 
(COP) 
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Annex  B:  List  of  Tools 


Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

@Mail  -  Atmail  Groupware 
solution 

Complete  Web  interface  for  Shared  Calendar,  Tasks  and  Address  book.  Ajax  interface,  multi¬ 
browser  compatible.  Includes  Outlook  Sync  utility  to  create  a  lightweight  Exchange  clone,  running 
under  Linux  or  Windows. 

Collaborative 

n/a 

Proprietary 

(@mail) 

24SevenOffice 

24SevenOffice  is  an  Enterprise  resource  planning  (ERP)  and  Customer  relationship  management 
(CRM)  system  for  small  and  medium  sized  businesses  delivered  on  demand  through  a  Ajax- 
powered  web-based  interface  (Software  as  a  Service).  The  system  includes  modules  for:  CRM, 
Accounting,  Invoicing/Ordering,  Procurement,  Email,  Calandar,  Project  Management,  Content 
Management  system  and  E-commerce. 

Collaborative 

Groupware,  web 
based  solutions 

Proprietary 

(24SevenOffice) 

4Team  for  Microsoft  Outlook 

Create  and  share  projects  workspaces  in  Outlook  with  or  without  an  Exchange  server. 

Collaborative 

n/a 

Proprietary 

(4Team 

Corporation) 

Access  Grid 

The  Access  Grid®  is  an  ensemble  of  resources  including  multimedia  large-format  displays, 
presentation  and  interactive  environments,  and  interfaces  to  Grid  middleware  and  to  visualization 
environments.  These  resources  are  used  to  support  group-to-group  interactions  across  the  Grid. 

Collaborative 

Real  Time  Audio, 

Video  and  Data 
Collaboration 

Open  Source 

ACE 

A  collaborative  real-time  text  editor 

Collaborative 

Groupware,  Other 

Open  Source 

Alfresco 

Content-management,  workflow,  and  portal.  Provides  document  management,  collaboration, 
records  management,  knowledge  management,  web  content  management  and  imaging. 

Collaborative 

Groupware,  Other 

Open  Source 

Defence  Knowledge  Online 
(DKO) 

Launched  in  2001,  Defence  Knowledge  Online  allows  soldiers  to  stay  connected.  DKO  offers 
email,  forums,  IM,  chat,  web  conferencing  and  application  sharing  as  well  as  giving  units  their  own 
working  space. 

Collaborative 

Mil/Gov 

Proprietary 
(Department  of 
Defense) 

Bamboo 

A  virtual  environment  toolkit  focused  on  the  ability  for  the  system  to  dynamically  configure  itself 
without  explicit  user  interaction,  enabling  the  system  to  take  on  new  functionality  after  execution. 
Enables  applications  to  be  dynamically  reconfigurable. 

FEDEP 

(Development) 

4,5 

Open  Source 

Bantu 

Bantu  Instant  Messaging  (IM)  and  Presence  Platform  provides  organizations  with  a  secure  Instant 
Messaging  technology  that  includes  one-to-one  and  group  messaging,  conference  rooms  with 
queuing,  logging  and  privacy  controls. 

Collaborative 

Mil/Gov 

Proprietary 

(Bantu) 

Basecamp 

Web  based  project  collaboration  and  management  tool  including  chat,  time  tracking  and  file 
sharing. 

Collaborative 

Mil/Gov 

Proprietary 

(37signals) 

Batipi  Work  Spaces 

Secure  online  collaborative  working  environment  that  includes  a  virtual  online  internet  accessible 
office  allowing  access  to  email,  sharing  and  editing  documents  with  others,  and  storing  files. 

Collaborative 

n/a 

Proprietary 

(Batipi) 

Bricolage 

An  open-source  enterprise-class  content  management  system  that  assists  with  creating, 
managing,  and  publishing  the  vast  libraries  of  content  essential  to  any  organization.  Includes  fully- 
configurable  workflows,  customizable  document  types,  multisite  management  capabilities,  and 
comprehensive  Perl-  and  PHP-based  templating  support. 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 
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Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

BSCW  Basic  Support  for 
Collaborative  Work 

Enables  collaboration  over  the  Web.  BSCW  is  a  'shared  workspace1  system  which  supports 
document  upload,  event  notification,  and  group  management.  To  access  a  workspace  you  only 
need  a  standard  Web  browser  as  the  product  is  free.. 

Collaborative 

n/a 

Proprietary 

OrbiTeam 

Software  GmbH  & 
Co.  KG) 

CDCIE  Portal 

Provide  a  secure  and  scalable  collaboration  tool  for  DOD  that  solves  the  tactical  chat,  cross 
domain,  full  functional  (minus  video)  collaboration  requirements 

Collaborative 

Mil/Gov 

Open  Source 

Central  Desktop 

Wiki-based  on  demand  team  collaboration  software  for  small  and  medium  sized  businesses. 

Collaborative 

n/a 

Proprietary 
(Central  Desktop) 

Citadel  (Groupware) 

Community-oriented  collaboration  software  combined  with  classic  e-mail  and  calendar  features 
that  are  focused  on  connecting  communities  of  people  together.  A  Citadel  system  is  made  up  of 
containers  called  “rooms.”  A  room  may  be  used  as  an  email  folder,  a  discussion  forum,  a  real-time 
chat,  a  mailing  list,  a  calendar,  an  address  book,  an  RSS  sink  or  sometimes  a  combination  of  any 
of  the  above.  Furthermore,  you  can  replicate  rooms  between  multiple  Citadel  nodes,  allowing  you 
to  set  up  a  federated,  distributed  messaging  environment. 

Collaborative 

Groupware,  classic 

client-server 

solutions 

Open  Source 

Collabnet 

Provides  solutions  for  distributed  collaborative  software  development. 

Collaborative 

n/a 

Proprietary 

(Collabnet) 

Collaborative  Virtual 
Workstation  (CVW) 

This  is  a  software  prototype  developed  by  MITRE  that  supports  a  collaborative  environment 
optimized  for  supporting  persistent,  geographically  dispersed  virtual  rooms.  CVW  provides  chat, 
audio/video  conferencing,  application  sharing,  electronic  whiteboarding,  and  multi-point 
communications.  At  the  time  this  paper  was  written,  MITRE  was  looking  for  a  vendor  who  would 
assume  responsibility  for  managing  and  improving  the  software 

Collaborative 

Mil/Gov 

Proprietary 

(MITRE 

Corporation) 

Collabworx 

CollabWorx  has  taken  a  unique  approach  to  providing  collaborative  solutions  with  its  CollabWorx 
Platform.  Building  collaboration  and  communication  solutions  on  this  platform  offers  benefits  over 
either  of  the  two  traditional  approaches:  implementing  a  packaged,  “out  of  the  box"  solution,  or 
implementing  a  custom-built  solution."  This  tool  is  used  by  the  Army  Training  Support  Center  in 

Fort  Eustis,  VA.  In  September  2005,  the  US  Army  DITSCAP  ATO  provided  CollabWorx  with 
certification  for  web-based  audio/videoconferencing 

Collaborative 

Mil/Gov 

Proprietary 

(Collabworx) 

Collanos 

An  open  source  alternative  to  Microsoft  Office  Groove 

Collaborative 

Project 

Collaboration,  web 
based  solutions 

Open  Source 

Collanos  Workplace 

Is  a  peer-to-peer  team  collaboration  desktop  tool. 

Collaborative 

n/a 

Proprietary 

(Collanos) 

CommunityZero. 

Online  community  systems  for  virtual  collaboration  and  knowledge  sharing. 

Collaborative 

n/a 

Proprietary 

(Ramius 

Corporation) 

Composable  FORCEnet 
(CFn) 

A  tool  for  expeditionary  operations  including  amphibious  assault  planning  and  sustainment  ashore 
of  the  follow-on  and  follow-up  echelons.  CFn  is  a  product  of  the  Space  and  Naval  Warfare 

Systems  Center,  San  Diego 

Collaborative 

Mil/Gov 

Proprietary 
(Department  of 
Defense) 
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Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

Coneix 

A  project  management  software 

Collaborative 

n/a 

Proprietary 

(Coneix) 

Confluence 

Is  an  enterprise  wiki  used  for  collaboration  and  knowledge  sharing. 

Collaborative 

Proprietary 

(Atlassian) 

Connect 

Highly-customizable  meeting,  training,  and  presentation  tools  are  provided  to  ease  the  creation, 
deployment  and  tracking  of  online  meetings,  trainings  and  on-demand  presentations. 

Collaborative 

n/a 

Proprietary 

(Adobe) 

ContactOffice 

Is  a  web  based  collaboration  tool  that  allows  users  to  set  up  ‘virtual’  offices. 

Collaborative 

n/a 

Proprietary 

(ContactOffice 

Group  S.A.) 

Croquet  project 

Powerful  open  source  software  development  environment  for  the  creation  and  large-scale 
distributed  deployment  of  multi-user  virtual  3D  applications  and  metaverses  that  are  (1)  persistent 
(2)  deeply  collaborative,  (3)  interconnected  and  (4)  interoperable.  The  Croquet  architecture 
supports  synchronous  communication,  collaboration,  resource  sharing  and  computation  among 
large  numbers  of  users  on  multiple  platforms  and  multiple  devices. 

Collaborative 

Groupware,  Other 

Open  Source 

Data 

Analysis  and  Reconciliation 
Tool  (DART) 

Regenerates  and  optimizes  existing  visual  terrain  databases  for  new  platforms;  can  create  new 
versions  based  on  how  new  sensors  would  “see"  the  terrain.  Can  be  used  to  map  and  match 
lexicon  items  as  appropriate. 

FEDEP 

(Development) 

3,4 

Unknown 

Data  Collection  Tool  (DCT ) 

After  action  review  (AAR)  tool  used  by  Defence  Modelling  and  Simulation  Office  (DMSO). 

FEDEP 

(Development) 

4,  5,  6,  7 

Proprietary 
(Department  of 
Defense) 

DEVS 

Discrete  Event  System  Specification  (DEVS)  is  a  framework  for  understanding  and  supporting  the 
activities  of  modeling  and  simulation,  based  on  generic  dynamic  systems  concepts. 

FEDEP 

(Development) 

4,5,6 

Unknown 

Digital  Dashboard 

A  digital  dashboard  is  a  customized  solution  for  knowledge  workers  that  consolidates  personal, 
team,  corporate,  and  external  information  and  provides  single-click  access  to  analytical  and 
collaborative  tools. 

Collaborative 

Mil/Gov 

Proprietary 

(Microsoft) 

DIS  Network  voice 

Provides  a  simulated  radio  model  with  shared  or  individual  radio  access  for  operators  located  on 
dispersed  network  nodes. 

FEDEP 

(Development) 

5,6 

Unknown 

DOORS 

Requirement  Management  Tool  that  contains  inherent  traceability  management  functions. 

FEDEP 

(Development) 

1,  2,3 

Proprietary 

(Telelogic) 

dotProject 

Open  source  project  management  software  that  aims  to  provide  the  project  manager  with  a  tool  to 
manage  tasks,  schedules,  communication  and  sharing. 

Collaborative 

Project 

Collaboration,  web 
based  solutions 

Open  Source 

EditGrid 

An  online  spreadsheet  with  access  control  and  revision  history  support,  whose  RTU  (real-time 
update)  feature  allows  multiple  users  to  collaborate  on  the  same  spreadsheet  simultaneously. 

Collaborative 

n/a 

Proprietary  (Team 
and  Concepts 
Limited) 

eGroupWare 

Enterprise  ready  group  ware  software  that  enables  the  management  of  contacts,  appointments,  to 
dos,  etc. 

Collaborative 

Groupware/Project 
Collaboration,  web 
based  solutions 

Open  Source 
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Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

eKM  (Enterprise  Knowledge 
Management) 

eKM  creates  a  shared  environment  for  disparate  organizations  that  have  geographically  dispersed 
locations.  It  is  a  web-based  collaborative  suite  of  knowledge  management  tools  used  to  capture 
and  manage  documents,  link  command  members  through  Communities  of  Practice  (CoP), 
manage  business  processes,  and  provide  ready  access  to  command  and  enterprise  information 
via  search  engines. 

Collaborative 

Mil/Gov 

Unknown 

Elluminate 

Web  based,  real-time  collaboration  tool  with  audio,  video,  whiteboard,  application  sharing 
simultaneously  between  Windows,  Macs,  and  Linux  users. 

Collaborative 

n/a 

Proprietary 

(Elluminate) 

Epiware 

Enterprise  document  management  tools  including  calendar,  wiki,  tasking/gantt  charts,  news  room, 
and  document  version/access  histories. 

Collaborative 

Project 

Collaboration,  web 
based  solutions 

Open  Source 
(Proprietary 
version  also 
available  from 
Epiware) 

Equater 

Scenario  Generation 

FEDEP 

(Development) 

4,5 

Unknown 

eRooms 

The  Office  of  Foreign  Disaster  Assistance  uses  eRooms  and  Abacus.  In  2002,  eRooms  was 
offered  as  part  of  the  NMCI  COTS  Catalogue  Contract  at  the  Space  and  Naval  Warfare  Systems 
Command.  The  monthly  service  cost  was  $32  per  seat.  See:  http://tinyurl.com/9sn6g  . 

Collaborative 

Mil/Gov 

Unknown 

Etazo 

Is  a  suite  of  products  based  on  a  comprehensive  online  platform  for  knowledge  workers  to 
communicate,  collaborate  and  innovate. 

Collaborative 

n/a 

Proprietary 
(Citadel  Rock 

Online 

Communities  Inc) 

everything  engine,  backing 
the  everything2  site 

The  Everything  Engine  is  an  open  source  content  management  system  written  in  Perl.  It  is 
designed  for  sites  which  allow  submissions  by  any  web  user,  but  can  be  configured  to  serve  as  a 
content  management  system  for  a  small  number  of  trusted  administrators.  It  has  a  very  database¬ 
centric  design;  each  page  in  the  Everything  Engine  is  a  node,  and  each  node  has  a  nodetype 
which  is  also  a  node;  much  of  the  code  is  stored  in  the  database.  This  simple  design  has  proved  to 
be  extremely  flexible. 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 

Exchange  Server  and  the 
Outlook  client 

Email  exchange  and  shared  calendar. 

Collaborative 

n/a 

Proprietary 

(Microsoft) 

exchange4linux 

A  collaborative  software  solution  that  provides  email-  and  webmail  server,  common  address  book, 
calendar,  notes  and  tasks.  It  is  designed  to  integrate  with  Microsoft  Office  Outlook  and  third-party 
tools  like  PDA-sync  or  Duplicates  Remover  for  Outlook.  It  is  meant  as  an  alternative  to  Microsoft 
Exchange  or  Microsoft  Small  Business  Server.  The  product  is  no  longer  available  for  download 
from  the  official  site.  Product  support  will  end  in  October  2007. 

Collaborative 

Groupware,  classic 

client-server 

solutions 

Open  Source 

FedDirector 

Part  of  HLA  Lab  Works,  provides  the  means  to  monitor  and  control  the  federation  execution. 

FEDEP 

(Development) 

4,5,6 

Unknown 

FedProxy 

Part  of  the  HLA  Lab  Works  suite,  can  debug  federate’s  HLA  interface,  perform  tests  of  the  RTI  & 
network,  and  even  provide  a  stand-in  for  missing  federates.  Part  of  HLA  Lab  Works  Suite  Tests 

HLA  interface. 

FEDEP 

(Development) 

4,5 

Unknown 
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Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

Federation  Execution 

Planning  Workbook  (FEPW) 

HLA  support  DMSO  tools 

FEDEP 

(Development) 

4,5 

Unknown 

FirstClass 

Provides  solutions  for  communication,  collaboration,  content  management,  and  online  networking. 

Collaborative 

n/a 

Proprietary 

(OpenText) 

FlashMeeting 

It  is  a  simple  but  sophisticated  web  based  'meeting'  tool,  allowing  a  group  of  people  to  setup  and 
have  a  meeting  with  each  other  using  the  internet.  FlashMeeting  has  a  complementary  product 
called  FlashMeeting  Memo,  which  supports  the  direct  reference  of  any  part  of  a  live  session 
recording  contributed  by  any  one  of  the  participants. 

Collaborative 

Mil/Gov 

Proprietary 
(Knowledge  Media 
Institute) 

FLSIM/HELISIM 

Reconfigurable  fixed  or  rotary  wing  high-fidelity  aero-model  which  can  integrate  with  any 
technology  which  HLA  enables  a  simulation. 

FEDEP 

(Development) 

4,5 

Unknown 

Federation  Management 

Tool  (FMT) 

HLA  support  DMSO  tools 

FEDEP 

(Development) 

5,6 

Unknown 

Federation  Verification  Tool 
(FVT) 

HLA  support  DMSO  tools 

FEDEP 

(Development) 

5,6 

Unknown 

Generic  Applications  Server 

Web  application  needs  (such  as:  wiki,  blog,  discussions,  surveys,  document  repositories,  feeds, 
faqs,  portals). 

Collaborative 

n/a 

Proprietary 
(Things  Prime) 

GERTICO 

Modular  RTI  based  on  CORBA. 

FEDEP 

(Development) 

4,  5,6 

Proprietary 

(Fraunhofer 

Institute) 

GL  Studio 

One  tool  in  the  category  of  rapid  application  development. 

FEDEP 

(Development) 

3,4 

Proprietary 

(DiSTI) 

Glance2 

Glance  is  a  real  time  desktop  sharing  tool  that  allows  up  to  fifteen  distributed  participants  to  see 
exactly  what  you  show  them  on  your  desktop.  They  do  not  need  to  download  anything;  they  have 
no  costs,  and  need  only  use  any  common  browser.  Although  designed  for  sales,  it  can  be  used 
for  sharing  any  information  or  data  that  needs  to  be  shared  within  a  small  distributed  group. 

Collaborative 

Mil/Gov 

Unknown 

Gobby 

Gobby  is  a  free  collaborative  editor  supporting  multiple  documents  in  one  session  and  a  multi-user 
chat.  It  runs  on  Microsoft  Windows,  Mac  OS  X,  Linux  and  other  Unix-like  platforms. 

Collaborative 

Editors 

Open  Source 

GoToMeeting 

Web  conferencing  and  collaboration  tool  that  facilitates  holding  effective  online  meetings,  training 
sessions  and  collaboration  gatherings. 

Collaborative 

Mil/Gov 

Proprietary  (Citrix 
Online) 

Groove 

A  peer-to-peer  collaboration  platform  allowing  users  the  ability  to  work  collectively  on  a  project. 
Multiple  users  can  log  on  together  or  work  individually  within  a  shared  space  that  contains  the 
information  (e.g.,  documents,  etc.)  that  they  are  working  on. 

Collaborative 

Project 

Collaboration,  web 
based  solutions 

Proprietary 

(Microsoft) 

GroupVille 

Web-based  Groupware  package  for  desktops  and  smartphones 

Collaborative 

n/a 

Proprietary 
(Sendai  Systems) 

Hexagon 

Simple  but  sophisticated  web  based  'presence'  tool  allowing  a  group  of  people  to  stay  in  touch  with 
each  other  within  a  private,  persistent,  virtual  'room'.  As  long  as  you  have  an  internet  connection 
you  can  join  'your'  Hexagon  community  anywhere  in  the  world 

Collaborative 

Mil/Gov 

Proprietary 
(Knowledge  Media 
Institute) 

GroupWise 

Cross-platform  collaborative  software  product  from  offering  e-mail,  calendaring,  instant  messaging 
and  document  management. 

Collaborative 

n/a 

Proprietary 

(Novell) 

DRDC  Ottawa  CR  2007-??? 


43 


Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

HFC  SDK 

The  HFC-SDK  1.0  included  the  HLA  Foundation  Class  (HFC)  Framework,  the  OMLib  Library,  the 
HFC  Automation  Tool  (HAT)  on  Windows  only,  and  an  HFC  rework  of  the  HelloWorld  sample 
federate  included  with  the  DMSO  RTI1.3v6.  The  HFC  provides  an  application  framework  for  HLA 
federates  in  much  the  same  way  the  Microsoft  Foundation  Class  (MFC)  library  provides  a 
framework  for  Windows  applications.  OMLib  offers  the  ability  to  dynamically  read  in  and 
manipulate  HLA  object  model  data  from  OMT-DIF  files.  The  HAT  automates  the  process  of 
mapping  HLA  object  model  content  to  C++  source  code  (providing  traceability)  through 
specialization  of  HFC  components.  HFC  SDK  1.3  is  the  current  update  to  HFC-SDK  1.0  and 
enables  development  of  HLA  federates  to  collaborate  with  the  DMSO  RTI  1.3v6. 

FEDEP 

(Development) 

1.3  4,5 

Unknown 

HLA  Control 

Has  the  functionality  of  the  standard  HLA  FEPW,  plus  full  lifecycle  federation  management 
capabilities.  Allows  users  to  plan  federation,  determine  if  performance  requirements  are  satisfied 
and  identify  and  correct  run  time  inaccuracies. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

HLA  Exercise  explorer 

A  fully  functional  HLA  Manager  Federate  designed  to  aid  in  the  development  of  HLA  Federates 
and  Federations.  The  Exercise  Explorer  provides  the  developer  with  critical  information  about  the 
current  running  state  of  an  HLA  Federation  Execution  including  run  time  information  on  each 
Execution  Member. 

FEDEP 

(Development) 

5,6 

Unknown 

HLA  Integration  Framework 

The  framework  software  provides  ready-made  use  of  many  HLA  functions  and  simpler  interfaces 
to  the  RTI. 

FEDEP 

(Development) 

5,6 

Unknown 

HLA  Results 

Is  a  comprehensive  data  management  system  with  all  the  features  needed  to  collect,  store  and 
understand  federation  data. 

FEDEP 

(Development) 

4,  5,  6,  7 

Unknown 

Holocene  Conversation 

Mode 

Real-time  online  communications  though  with  a  unique  interface,  the  designers  argue  that  the  tool 
provides  a  much  better  situation  awareness  than  other  chat  tools.  Takes  advantage  of  the 
observation  that  human  beings  utilize  a  number  of  real  world  characteristics  to  participate  in, 
perceive,  control,  and  glean  subtleties  from  conversations. 

Collaborative 

Mil/Gov 

Unknown 

Horde 

PHP-based  suite  of  web  applications,  including  a  webmail  program,  contact  manager,  calendar 
manager,  etc. 

Collaborative 

Project 

Collaboration,  web 
based  solutions 

Open  Source 

HP  Openview  Network  Node 
Manager 

Local  area  and  wide  area  network  management  tool. 

FEDEP 

(Development) 

5,6 

Unknown 

Hula  mail  and  calendar 
based  collaboration  seeded 
by  Novell 

Community  project  to  create  an  open  source  collaboration  server 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 

Hummingbird 

Provides  a  highly  secure,  Web-based  collaborative  workspace  for  dispersed  teams  across  and 
beyond  the  enterprise. 

Collaborative 

Mil/Gov 

Proprietary 

(OpenText) 

Hyperwave 

Collaborative  Information  Management  (CIM) 

Collaborative 

Mil/Gov 

Proprietary 

(Hyperwave) 

Ibis  Model  Editor 

Ibis  Model  Editor  is  a  CAFDE-compliant  software  package  designed  to  create  HLA-compliant 
models.  Model  Editor  is  still  in  a  beta,  not  final,  stage.  As  such  it  may  not  be  as  refined  as  a  final 
product  would  be.  Trial  copies  may  be  downloaded  for  evaluation  purposes  only. 

FEDEP 

(Development) 

3,  4,  5,  6 

Proprietary  (ibis 
Research 

Corporation) 
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Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
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Open  Source/ 
Proprietary 
(Vendor) 

Ibis  RTI  Adapter 

Ibis  RTI  Adapter  is  an  ActiveX  component  that  exposes  DoD’s  HLA  (High  Level  Architecture)  RTI 
(Run  Time  Infrastructure)  to  COM/ActiveX  applications. 

FEDEP 

(Development) 

3,  4,  5,  6 

Proprietary  (ibis 
Research 

Corporation) 

Information  Workspace 
(IWS) 

A  Web-based,  collaboration  environment  featuring  virtual  rooms,  audio/video  conferencing,  chat, 
electronic  whiteboarding,  and  application  sharing  with  multipoint  communications 

Collaborative 

Mil/Gov 

Proprietary 

(General 

Dynamics) 

InfoWorkSpace 

Provides  teams  that  are  geographically  dispersed  with  the  ability  to  collaborate  and  share 
information  in  a  real-time  virtual  environment.  Provides  organizations  the  tools  to  communicate 
through  a  variety  of  options,  conferences,  and  shared  applications. 

Collaborative 

n/a 

Proprietary 

(Ezenia) 

Interdaptor 

Provides  a  customizable  out  of  the  box  solution  to  you  simulation  interoperability  needs.  Provides 
true  interoperability  between  HLA,  DIS,  and  customized  interfaces  or  protocols;  achieves  cost- 
effective  HLA  compliance;  and  allows  interoperability  between  legacy  and  other  systems. 

FEDEP 

(Development) 

4,5,6 

Unknown 

Intersim 

InterSIM  software  enables  simulations  and  instrumented  systems  to  be  networked  together  in  the 
same  synthetic  environment  according  to  DIS  IEEE  1278.1-1995  or  HLA  standards.  HLA  RTI 
specs  1.1  with  DMSO  RTI  1.0. 

FEDEP 

(Development) 

5,6 

Unknown 

ITEMS 

ITEMS  provides  simulation  and  CGF  capabilities;  see  also  STRIVE.  Liteflite  3, 4  LiteFlite  TM  Re- 
Configurable  Simulation  Toolkit  Is  Low-Cost,  PCBased  Solution  Providing  Photo-Realistic  Geo- 
Specific  Dynamic  Environments. MAK  Data 

FEDEP 

(Development) 

4,  5,6 

Unknown 

iUpload  (also  called 
Awareness) 

Tight  integration  between  blogs  and  content  management,  allowing  users  to  take  full  advantage  of 
the  blog  phenomenon  at  a  corporate  level  to  connect  and  stay  connected  with  employees, 
customers,  partners  or  other  key  constituencies. 

Collaborative 

Mil/Gov 

Proprietary 

(Awareness) 

Jahia 

Content  management,  corporate  portal,  document  management,  and  collaboration  suite 

Collaborative 

Groupware,  Other 

Open  Source 

JotSpot  Wiki 

Supports  calendars,  spreadsheets,  file  repositories,  and  photo  galleries.  Built  for  ease  of  use. 
JotSpot  identifies  itself  as  an  application  wiki.  One  can  create  dynamic  tables,  and  attach  any  type 
of  file 

Collaborative 

Mil/Gov 

Proprietary 

(Google) 

Kolab 

An  open  source  groupware  suite.  It  consists  of  the  Kolab  server  and  a  wide  variety  of  Kolab 
clients,  including  KDE  PIM-Suite  Kontact  (full  functionality  since  KDE  3.4.1),  Horde  Web  frontend 
(currently  in  late  beta  status),  Mozilla  Thunderbird  and  Mozilla  Lightning  with  SyncKolab  extension 
(beta)  and  Microsoft  Outlook  with  Toltec  Connector  (stable)  or  Konsec  Konnektor  (stable).  The 
special  idea  behind  Kolab  is  the  usage  of  IMAP  as  an  underlying  protocol  not  only  for  email,  but 
also  for  contact  and  calendar  entries. 

Collaborative 

Groupware,  classic 

client-server 

solutions 

Open  Source 

Kwiki 

Wiki  software  that  is  especially  popular  in  the  Perl  community. 

Collaborative 

Wiki  collaborative 
software 

Open  Source 

Livelink 

Comprehensive  Web  content  management  solution  that  enables  organizations  to  create  and 
manage  content  once  and  re-use  it 

Collaborative 

n/a 

Proprietary 

(OpenText) 

Logger 

The  MAK  Data  Logger  is  a  system  for  capturing  and  relaying  simulation  data. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

Live  Communications  Server 

Instant  messaging  (IM)  and  presence  software. 

Collaborative 

n/a 

Proprietary 

(Microsoft) 
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Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

Live  Meeting 

Hosted  Web  conferencing  service. 

Collaborative 

n/a 

Proprietary 

(Microsoft) 

Lotus  Notes  and  Domino 

Security-rich  e-mail,  calendaring  and  scheduling,  instant  messaging  and  support  for  a  wide  range 
of  business  applications. 

Collaborative 

n/a 

Proprietary  (IBM) 

Lotus  QuickPlace 

A  workspace  on  the  Web  for  team  collaboration  among  IBM  employees  and  customers,  suppliers 
and  Business  Partners. 

Collaborative 

n/a 

Proprietary  (IBM) 

MAK  Gateway 

The  MAK  Gateway  translates  DIS  PDUs  into  RTI  service  invocations,  and  vice  versa,  in  real-time. 

FEDEP 

(Development) 

5,6 

Unknown 

MAK  PVD 

Provides  Multiple  Map  Views,  Controls  Stealth,  Calculates  Line-Of  -Sight,  Displays  Contours  and 
Grid  Lines,  Language  Independent,  Extensible  Through  Plug-In  Interface,  FOM-Agile  Through 
VRLink’s  FOM-Mapper  Architecture. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

MAK  Real-time  RTI 

No  RTI  executive  or  other  central  server  is  necessary  to  use  the  MAK  RTI,  making  initialization 
quick  and  easy.  It  can  be  configured  to  use  point-to-point,  broadcast,  or  multicast  communications 
for  maximum  flexibility  across  different  network  architectures.  Optimized  for  realtime  simulations. 

FEDEP 

(Development) 

5,6 

Unknown 

MAK  Stealth 

Used  for  3D  visualization,  situation  awareness,  debugging  a  simulation,  or  after-action  review. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

MAK  VR  Forces 

CGF  Mak  tools.  Not  a  tool  to  support  process  but  a  possible  federate. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

Marratech 

Realtime  collaboration  with  audio,  video,  whiteboard  and  chat. 

Collaborative 

Proprietary 

(Google) 

MediaWiki 

A  free  software  wiki  package  originally  written  for  Wikipedia. 

Collaborative 

Wiki  collaborative 
software 

Open  Source 

Meebo 

Meebo  tool  provides  IM  capability  across  various  IM  services  and,  being  web-based  unlike  most 

IM  clients,  can  be  accessed  from  any  computer,  not  just  one’s  office  or  home  computer. 

Collaborative 

Mil/Gov 

Unknown 

MERBoard 

Intended  for  use  as  a  collaboration  tool  within  a  corporate  environment  to  support  fast  encounters 
and  spontaneous  meetings,  it  is  also  used  as  a  multi-mission  platform  for  collaborative  mission 
control  applications  for  NASA 

Collaborative 

Mil/Gov 

Unknown 

MERIT 

A  powerful,  web-enabled  tool  that  graphically  depicts  the  current  Marine  Corps  readiness  posture 
and  detailed  supply  and  maintenance  information  using  emerging  data  visualization  techniques. 
MERIT  transforms  data  into  information  that  provides  a  dynamic  and  adaptable  view  of  equipment 
readiness  by  commodity,  functional  area,  and  organization 

Collaborative 

Mil/Gov 

Unknown 

MindManager 

MindManager  6  is  the  market  leading  productivity  software  for  visualizing  and  managing 
information,  allowing  individuals  and  teams  to  more  effectively  think,  plan,  and  collaborate. 

Collaborative 

n/a 

Proprietary 

(Mindjet) 

ModlOS  2D  PVD  4, 

The  Plan  View  Display  (PVD)  is  one  application  in  the  ModlOS  tool  suite.  It  provides  a  2D  view  of 
the  simulation  and  configurable  icons.  Designed  for  DIS  and  included  HLA  gateway. 

FEDEP 

(Development) 

5,6 

Unknown 

ModlOS  3D  Stealth  Viewer 

The  Stealth  Viewer  is  one  application  in  the  ModlOS  tool  suite.  It  provides  a  3D  display  of  the 
battlefield  from  various  points  of  view  (cockpit,  independent,  etc.)  Supports  smoothing  of  entity 
positions,  special  effects  such  as  explosions,  and  atmospheric  effects.  Designed  for  DIS  and 
included  HLA  gateway. 

FEDEP 

(Development) 

4,  5,6 

Unknown 
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Tool 
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Collaborative  or 
FEDEP  Focus 
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FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

ModlOS  AAR 

The  After  Action  Review  (AAR)  is  one  application  in  the  ModlOS  tool  suite.  It  provides  a  data 
logging  and  replay  facility,  automatic  generation  of  performance  reports,  and  remote  control  of  the 

2D  PVD  and  3D  Stealth  Viewer.  Designed  for  DIS  and  included  HLA  gateway. 

FEDEP 

(Development) 

5,  6,7 

Unknown 

ModlOS  Exercise  Controller 

The  Exercise  Controller  is  one  application  in  the  ModlOS  tool  suite.  It  provides  configurable  control 
of  simulation  applications,  including  2D  and  3D  displays,  computer-generated  forces,  etc.  It  is  used 
to  start,  resume,  stop  and  freeze  simulations,  generate  reports,  create  and  remove  entities,  etc. 
Designed  for  DIS  and  included  HLA  gateway. 

FEDEP 

(Development) 

5,6 

Unknown 

ModlOS  logger/player 

The  Logger/Player  is  one  application  in  the  ModlOS  tool  suite.  It  provides  a  data  logging  and 
replay  facility.  Designed  for  DIS  and  included  HLA  gateway. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

ModlSE 

Framework  that  facilitates  composition  of  and  interoperability  among  interactive  simulation 
applications.  It  includes  a  web-based  model  repository,  a  GUI  and  a  run-time  Interoperability 
engine.  ModlSE  stands  for  Modular  Interoperable  Synthetic  Environment. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

ModSAF 

Computer  Generarated  Forces  (CGF)  creation.  It  is  not  a  tool  to  support  process  but  a  possible 
federate.  Retired  software  that  is  being  replaced  by  OneSAF  Test  Bed  version  1.0. 

FEDEP 

(Development) 

4,  5,6 

Proprietary 
(Department  of 
Defense) 

Multigen 

Products  that  help  to  collect,  rectify,  build,  and  publish  data  for  simulations  involveing  complex 
real-time  3D  environments 

FEDEP 

(Development) 

2  3,4 

Proprietary 
(Presagis,  through 
CAE) 

Multigen  Creator 

Creator  is  a  comprehensive  toolset  for  the  rapid  generation  of  optimized  object  models,  high 
fidelity  terrain  and  synthetic  environments  for  use  in  realtime  3D  visual  simulation,  simulation 
based  training,  and  urban  simulation. 

FEDEP 

(Development) 

3,4 

Proprietary 
(Presagis,  through 
CAE) 

Multigen  Creator  -Sedris 
export 

MultiGen  SEDRIS  Exporter  is  a  plug-in  for  Creator  that  provides  interoperation  technology  for  the 
defense  training  and  simulation  community.  The  SEDRIS  Exporter  is  a  flexible,  user-guided 

SEDRIS  database  production  solution  that  supports  STRICOM  and  DMSO’s  Synthetic 
Interoperability  Strategy  in  an  easy-to-manage  procedural  workflow.  The  SEDRIS  Exporter 
translates  industry  standard  3D  OpenFlight  files  into  the  SEDRIS  Transmittal  Format  (.stf),  making 
this  an  invaluable  tool  for  any  project  with  SEDRIS  data  requirements. 

FEDEP 

(Development) 

3,4 

Proprietary 
(Presagis,  through 
CAE) 

My  Teamwork 

Is  a  web-based  collaboration  tool  supporting  presence,  instant  messaging,  audio  and  video 
conferencing  and  application  sharing 

Collaborative 

n/a 

Proprietary 

(Alcatel) 

MyWebDesktop 

Personal  and  collaborative  desktop  on  the  internet. 

Collaborative 

n/a 

Proprietary 

(MyWeb 

Desktop.net 

Netmeeting 

A  Microsoft  product  that  supports  point-to-point  communications  for  its  audio/video  conferencing, 
chat,  application  sharing,  and  electronic  whiteboarding 

Collaborative 

n/a 

Proprietary 

(Microsoft) 

Nuxeo  Collaborative  Portal 
Server 

Content  management  and  collaborative  platform  based  on  Zope 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 

Office.com 

A  Web-based  integrated  suite  of  15  applications  that  includes  group  calendaring,  document 
management,  contact  management,  email,  wiki  and  other  collaborative  tools. 

Collaborative 

n/a 

Proprietary 

(Office.com) 
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Office  Gateway 

Online  collaboration  suite  for  that  uses  the  internet  to  allow  teams  to  work  both  remotely 
(accessing  the  office  from  anywhere)  and  together  (sharing  schedules,  documents  and 
information).  Company  based  in  Atlantic  Canada. 

Collaborative 

n/a 

Proprietary  (Alt 
Linus) 

Object  Model  Development 
Tool  (OMDT) 

DMSO  tool  that  automates  the  process  of  constructing  SOMs  and  FOMs 

FEDEP 

(Development) 

3,  4,5 

Unknown 

OMDT  Pro 

Editor  for  creation  and  modification  of  SOMs  and  FOMs. 

FEDEP 

(Development) 

3,  4,5 

Unknown 

Omni  4, 

Part  of  the  HLA  Lab  Works  suite,  used  to  Integrate  simulations  Omni  is  a  set  of  related  software 
components  and  applications  that  together  give  simulations  the  ability  to  establish  a  Federation 
Object  Model  (FOM)  independent  interface  to  the  HLA  Runtime  Infrastructure  (RTI).  Part  of  the 

HLA  Tool  Suite  Middleware  used  to  integrate 

FEDEP 

(Development) 

5 

Unknown 

OneSAF  Testbed 

Used  in  the  creation  of  Computer  Generated  Forces. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

OpenGroupware.org 

Their  goal  is  to  create  a  groupware  server  to  integrate  the  leading  open  source  office  suite 
products 

Collaborative 

Groupware,  classic 

client-server 

solutions 

Open  Source 

OpenLink  Wiki  Engine 

A  distributed  collaborative  suite  of  applications  that  includes:  Weblog  Engine,  Wiki  Engine, 

Bookmark  Manager,  Discussion/Conversation  Services,  Feed  Aggregation,  Photo  Sharing,  and 
Unified  Data  Storage  etc.  It  includes  realm  specifc  data  access  via  SPARQL,  GData,  OpenSearch, 
XQuery,  XPath,  and  Full  Text  in  general. 

Collaborative 

Wiki  collaborative 
software 

Open  Source 

Open-Xchange 

Email,  calendar,  contacts,  tasks  and  documents  are  includes  as  part  of  its  11  collaboration 
modules.  Integrates  with  PDA's. 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 

Ourmedia 

A  project  allowing  any  person  with  net  access  to  publish  their  text,  image,  audio  and/or  video  files 
for  public  consumption. 

Collaborative 

Mil/Gov 

Open  Source 

PabloDraw 

Collaborative  text  and  ANSI/ASCII  art  editing  on  Windows 

Collaborative 

n/a 

Open  Source 

phpGroupWare 

A  multi-user  groupware  suite  written  in  PHP  that  provides  about  50  web-based  applications 
(including  Calendar,  Address  Book,  an  Advanced  Projects  Manager,  Todo  List,  Notes,  Email, 
Newsgroup-  and  Headlines  Reader,  a  File  Manager).  The  system  as  a  whole  supports  user 
preferences,  themes,  user  permissions,  multi-language  support  and  user  groups. 

Collaborative 

Groupware,  classic 
client-server 
solutions/web  based 
solutions 

Open  Source 

PHProjekt 

PHProjekt  is  a  modular  application  for  the  coordination  of  group  activities  and  to  share  information 
and  documents  via  the  web.  Components  include  Group  calendar,  project  management,  time  card 
system,  file  management,  contact  manager,  mail  client.  Etc. 

Collaborative 

Project 

Collaboration,  web 
based  solutions 

Open  Source 

ProjectDox 

Construction  Project  Management  and  Collaboration  Software 

Collaborative 

n/a 

Proprietary 

(Avolve) 

pRTI 

Pitch's  portable  Runtime  Infrastructure  (pRTI)  is  a  platform  independent  software  that  provides 

HLA  services  used  by  federates  to  co-ordinate  their  operations  and  data  exchange  during  an  HLA 
federation  execution.  pRTI  implements  all  services  documented  in  the  HLA  Interface  Specification 
vl.3. 

FEDEP 

(Development) 

5,6 

Unknown 
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pRTI  for  IEEE  1516 

The  product  implements  the  entire  1516  standard.  First  commercial  IEEE  1516  RTI. 

FEDEP 

(Development) 

5,6 

Unknown 

PSI-SA  3 

User  friendly  API  to  RTI.  Stresses  the  modelling  aspect. 

FEDEP 

(Development) 

4,5 

Unknown 

RAL  Wrapper 

RTI  Abstraction  Layer  for  C++  developed  simulation.  Facilitate  design,  allow  automatic  generation 
of  code  and  execution. 

FEDEP 

(Development) 

4,5 

Unknown 

Rally  Point 

A  web  based  HomeOwners  Association  (HOA)  software  designed  to  manage  files,  notes,  cars,  for 
people  who  live  together 

Collaborative 

n/a 

Proprietary  (WBP 
Systems) 

R-CAST 

R-CAST  supports  collaborative  activities  among  teammates  comprised  of  both  humans  and 
software  systems. 

Collaborative 

Mil/Gov 

Unknown 

RealDB 

Realistic  up-to-date  models.  Canadian,  Russian,  and  U.N.  army  equipment  visual  models,  3  levels 
of  detail  plus  damage  states. 

FEDEP 

(Development) 

3,4 

Unknown 

S2Focus 

Provides  exercise  management  tools,  including  a  Mission  Planner,  Recorder,  Manager,  Viewer, 
and  Analyzer. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

SAIDA 

Security  extensions  to  the  RTI  prototype  (CERTI)  developed  at  ONERA  (Office  National  d'Etudes 
et  de  Recherches  Aerospatiales).  These  extensions  are  aimed  at  guaranteeing  secure 
interoperation  of  simulations  belonging  to  various  mutually  suspicious  organizations.  It  is  an  UK/F 
cooperation. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

SameTime 

A  Lotus  product  that  interfaces  with  most  Web  browsers  and  provides  audio/video  conferencing, 
chat,  application  sharing,  electronic  whiteboarding,  and  awareness  with  multipoint  communications 

Collaborative 

Mil/Gov 

Proprietary  (IBM) 

Scalix 

Linux  email,  calendaring  and  messaging  platform. 

Collaborative 

Groupware,  classic 

client-server 

solutions 

Proprietary 

(Xandros) 

Scoop 

Scoop  is  a  “collaborative  media  application"  that  is  somewhere  between  a  content  management 
system,  a  web  bulletin  board  system,  and  a  weblog.  Scoop  is  designed  to  enable  your  website  to 
become  a  community  by  empowering  visitors  to  be  the  producers  of  the  site,  through  contributing 
news  and  discussion. 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 

Sedris  tools 

A  synthetic  environment  data  interchange  programme. 

FEDEP 

(Development) 

3,4 

Unknown 

Sequoia  Integrator  for  HLA 

The  Integrator  provides  the  means  to  rapidly  integrate  new  or  existing  simulation  systems  into  HLA 
environments.  SEQUOIA  Integrator  for  HLA  vl.O  is  currently  available  on  Windows  NT©  for  use 
with  RTI1.3NG-V3. 

FEDEP 

(Development) 

4,5 

Unknown 

SGT 

Scenario  generation  HLA  lab  works. 

FEDEP 

(Development) 

3,4 

Unknown 

ShareDirect 

ShareDirect  lets  you  connect  any  folders  on  your  PC  to  any  number  of  users  instantly  and 
securely. 

Collaborative 

n/a 

Proprietary  (Share 
Direct) 

ShareO  for  Microsoft 

Outlook 

Share  Outlook  calendar,  contacts,  journal,  mail,  tasks  and  notes  folders  as  well  as  documents  and 
files  with  other  Outlook  users  without  a  server. 

Collaborative 

n/a 

Proprietary 

(4Team 

Corporation) 
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SharePoint  Services  and 
Microsoft  SharePoint  Portal 
Server 

Microsoft's  integrated  suite  of  server  capabilities  for  enterprise  search,  content  management, 
business  process  facilitation,  simplified  information  sharing,  and  enhanced  business  insight. 

Collaborative 

n/a 

Proprietary 

(Microsoft) 

Simple  Groupware 

A  groupware  package  written  in  PHP  that  uses  the  MySQL  database  (version  4  or  higher).  It 
contains  a  calendar  system,  an  email  client,  an  inventory  system,  and  a  number  of  other  features. 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 

Simplex  3 

The  main  design  concept  behind  the  HLA-interface  of  Simplex  3  is  to  hide  all  HLA-functionality 
from  the  model  developer.  It  should  be  noted,  that  this  approach  leaves  the  entire  model 
description  of  Simplex  3  models  unchanged,  no  matter  if  they  act  as  stand-alone  models  or 
federates  in  the  sense  of  HLA.  With  that  kind  of  an  HLA  interface  the  entire  interoperability  issue  is 
part  of  the  simulation  system  itself,  and  thus  not  part  of  the  simulation  model.  One  the  one  hand 
this  facilitates  the  re-use  of  existing  models,  and  on  the  other  hand  the  developer  of  new  models 
does  not  need  to  have  additional  knowledge  for  building  HLA-compliant  models. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

SIMplicity 

SIMplicity  is  an  integrated  development  environment  (IDE)  that  enables  developers  and  scientists 
to  rapidly  assemble  component  based  HLA  simulations  from  new  and  pre-existing  components  in 
a  visual  environment.  SIMplicity  assists  the  developer  throughout  the  development  life  cycle,  from 
design  to  development,  deployment  and  execution.  SIMplicity  uses  a  template-driven  code 
generation  process  to  create  all  of  the  simulation  entities  for  the  targeted  platform  specific 
simulation  model  (PSM). 

FEDEP 

(Development) 

4,  5,6 

Proprietary 

(Calytrix) 

Simulation  Support 
Environment  DUCTOR 

DUCTOR  is  an  architecture  which  allow  to  develop  operational  simulations  running  stand-alone  or 
as  an  HLA  federate.  It  is  00  (UML  based)  and  promotes  re-use  of  scenarios,  specific  behaviours 
and  platforms. 

FEDEP 

(Development) 

3,  4,  5,  6 

Unknown 

Simulation  Support 
Environment  ESCADRE 

Encapsulate  and  hide  low  level  HLA  interface  functionality,  providing  high  level  services  for  HLA 
interoperability.  Provide  an  00  methodology  and  a  tool  set  to  design,  implement  and  run 
standalone  simulations  and  HLA  federates. 

FEDEP 

(Development) 

3,  4,  5,  6 

Unknown 

Skopeo  Animation  System 

In  order  to  run  in  a  distributed  environment,  Skopeo  was  extended  for  HLA.  This  extension  uses 
the  Beta  release  of  the  Java  RTI  API  from  DMSO.  It  is  written  in  Java  and  runs  stand-alone  or  as 
applet  in  any  javacapable  web  browser.  Skopeo  was  enhanced  for  3D  animation  using  VRML2. 
Additionally  CORBA  mechanisms  are  used  for  communication  between  the  Skopeo  Applet  and  the 
Skopeo  server. 

FEDEP 

(Development) 

4 

Unknown 

Skype 

Skype  is  a  program  for  making  free  calls  over  the  internet  to  anyone  else  who  also  has  Skype. 
SkypeOut  allows  Skype  users  to  call  traditional  telephone  numbers,  including  mobile  telephones, 
for  a  small  fee. 

Collaborative 

Mil/Gov 

Proprietary  (Skype 
Limited) 

SlashCode  software 

The  open  source  collection  of  Perl  modules  and  stand-alone  programs  which  runs  Slashdot,  one 
of  the  oldest  and  most  popular  collaborative  weblogs. 

Collaborative 

Groupware,  web 
based  solutions 

Open  Source 

SLX  Simulation  Environment 

HLA  interface  provided  SLX  is  a  discrete  event  simulation  tool  for  the  Windows  95/98/NT  operating 
systems.  It  is  a  simulation  language  oriented  tool.  The  SLX  user  is  provided  with  an  interface  to 
the  RTI  and  the  possibility  of  “doing"  distributed  simulation  based  on  HLA  without  having  to  deal 
with  the  lowest  API-level  of  HLA. 

FEDEP 

(Development) 

4,5 

Unknown 
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SmartFED 

SmartFED  has  been  designed  to  operate  on  simulations  based  on  the  High  Level  Architecture 
(HLA),  developed  by  the  US  Defense  Modeling  and  Simulation  Office  (DMSO). 

FEDEP 

(Development) 

4,  5,6 

Unknown 

SmartMeeting 

SmartMeeting  is  a  conferencing,  collaboration  and  live  presentation  system  that  utilizes  a  new  type 
of  online  meeting:  a  3-D,  fully-immersive,  voice-enabled  with  spatial  3D  sound,  avatar-driven, 
multi-featured  virtual  office  space. 

Collaborative  -  CM 

Mil/Gov 

Proprietary 

(Convenos) 

SMOC 

A  standard  interface  to  HLA  for  developers  of  models  and  simulations.  Serves  as  a  DIS/HLA 
gateway  to  avoid  expensive  modifications  to  DIS-compliant  systems. 

FEDEP 

(Development) 

5,6 

Unknown 

Socialtext 

As  the  first  wiki  company,  Socialtext  is  the  leader  in  making  web  collaboration  secure,  scalable 
and  easy  to  use 

Collaborative 

Mil/Gov 

Proprietary 

(Socialtext 

Incorporated) 

SPEEDES 

A  simulation  engine  allows  the  simulation  builder  to  perform  optimistic  parallel  processing  on  high 
performance  computers,  networks  of  workstations,  or  combinations  of  networked  computers  and 
HPC  platforms. 

FEDEP 

(Development) 

4,5,6 

Open  Source 

Spider 

Although  primarily  a  resource  tool,  it  does  support  37  discussion  links  that  include  chat  rooms,  and 
23  listservs  links. 

Collaborative 

Mil/Gov 

Proprietary 

(Human 

Performance 

Centre) 

STAGE  Scenario 

Simplifies  the  building  of  a  tactical  database  to  then  simulate  dynamic,  interactive,  complex,  and 
real-time  tactical  or  operational  environments. 

FEDEP 

(Development) 

4,  5,6 

Proprietary 
(Presagis,  through 
CAE) 

STRIVE 

Includes  commercial  off-the-shelf  simulation  development  environment  and  a  high  fidelity,  full 
function  synthetic  tactical  environment  and  computer  generated  forces  package.  Helps  shorten 
development  cycles  by  allowing  developers  to  focus  on  building  models  without  the  concern  for 
how  the  models  communicate,  interact,  and  perform  in  real-time. 

FEDEP 

(Development) 

4,  5,6 

Proprietary  (CAE) 

SubEthaEdit 

A  powerful  and  lean  collaborative  text  editor 

Collaborative 

n/a 

Proprietary 

(TheCodingMonke 

ys) 

Synapse  Corporate 

Solutions 

Web-Based  Business  Software 

Collaborative 

n/a 

Proprietary 

(Synapse 

Corporate 

Solutions) 

TalkAndWrite 

TalkAndWrite  is  a  freeware  whiteboard  fast  and  simple  powered  by  Skype.A  Realtime 
collaboration  tool. 

Collaborative 

n/a 

Proprietary 

(TalkAndWrite) 

TeamLeader 

Away  to  manage  outsourced  teams.  Includes  workshops,  task  management,  discussions, 
document  handling,  status  reporting  etc. 

Collaborative 

n/a 

Proprietary 

(Process-one) 

TeamLinks 

Fully  integrated  presence  awareness,  IM,  VoIP,  whiteboard,  instant  meetings  and  collaboration 
tools,  completely  secure  across  domains 

Collaborative 

n/a 

Proprietary 

(Imera) 

TeamWork  Live 

TeamWork  Live  is  a  hosted  web-based  project  management  and  team  collaboration  tool  by 
CollectiveSoft. 

Collaborative 

n/a 

Proprietary 
(TeamWork  Live) 
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TeamPage 

Traction's  model  focuses  on  collaboration  within  and  among  groups  using  a  sophisticated 
permission  model  that  aggregates  multiple  blog/wiki  spaces  using  a  web-based  Journal  modeled 
on  the  principals  of  Douglas  Engelbart's  On-Line_System  (NLS). 

Collaborative 

n/a 

Proprietary 
(Traction  Software 
Inc) 

Telestra  HLA 

Supports  execution,  Remote  HLA  Management,  Radio  Simulation  and  Communications. 

FEDEP 

(Development) 

4,  5,6 

Unknown 

Terra  Vista/Terra  Vista  Pro 
Builder 

Used  to  create  visual  terrain  databases  in  OpenFlight  or  TerraPage  formats.  ProBuilder  version 
intended  for  “power  users”.  Both  versions  are  extensible. 

FEDEP 

(Development) 

3,4 

Unknown 

TerraTools 

Terrain  DB  Construction  Tool. 

FEDEP 

(Development) 

3,4 

Unknown 

Thinkature 

Thinkature  brings  rich  communication  to  the  web  by  combining  an  instant  messaging  system  with 
shared,  visual  workspace. 

Collaborative 

n/a 

Proprietary 

(Thinkature) 

TikiWiki 

A  full  featured,  multilingual,  Wiki/Content  Management  Services  (CMS)/Groupware  written  in  PHP 
and  maintained  by  an  active  and  international  community  of  benevolent  contributors. 

Collaborative 

Wiki  collaborative 
software 

Open  Source 

Tracker  Suite 

Tracker  Suite  leverages  the  Lotus  Notes  /  Domino  email  platform  to  provide  modular,  Web- 
enabled  applications  for  project  management,  help  desk,  personnel  management,  time  and 
expense  reporting,  purchasing,  asset  tracking,  CRM,  sales  force  automation  and  business 
reporting. 

Collaborative 

n/a 

Proprietary 

(Automation 

Centre) 

TrackerOffice 

TrackerOffice  leverages  the  Microsoft  Outlook  /  Exchange  email  platform  to  provide  tools  for 
project  management,  invoicing,  time  and  expense  reporting,  purchasing  and  business  reporting. 

Collaborative 

n/a 

Proprietary 

(Automation 

Centre) 

TUTOS 

A  tool  to  manage  the  organizational  needs  of  small  groups,  teams,  departments  in  project 
management  and  development  that  includes:  a  calendar  for  users  and  groups;  address  manager 
for  people,  companies  and  departments;  bug  tracking  system;  product/project  repository; 
mailboxes;  timetracking  on  projects,  installations  and  bugs;  Invoices;  Watchlists  to  stay  informed 
on  changes  on  projects/bugs  etc  via  email;  support  for  teams  that  are  distributed  over  different 
timezones;  color  themes  /  layout;  fine  grained  permission  handling;  change  history. 

Collaborative 

Groupware/Project 
Collaboration,  web 
based  solutions 

Open  Source 

TWiki 

Enterprise  wiki,  enterprise  collaboration  platform  and  knowledge  management  system 

Collaborative 

Wiki  collaborative 
software 

Open  Source 

UOB  DAT 

MSIAC  Web  Page;  Support  Exercises  Composition.VAPS  4  Rapid  prototyping  of  complex  human 
computer  interfaces;  generates  C-Code  which  can  be  HLA  enabled  using  any  HLA  integrator 
product. 

FEDEP 

(Development) 

2,3,4 

Unknown 

VEGA  3, 

Vega  Prime  is  a  software  environment  for  the  creation  and  deployment  of  realtime  visual 
simulation,  virtual  reality,  sensor  and  general  visualization  applications.  Vega  Prime  combines 
simulation  functionality  with  tools  to  create  an  infrastructure  to  build,  edit  and  run  sophisticated 
applications. 

FEDEP 

(Development) 

4,5 

Unknown 

VIP  Task  Manager 

Client/server  collaboration  software  that  allows  authorized  users  to  simultaneously  access  and 
share  common  database  through  Local  Network  (LAN)  to  see,  add  and  assign  tasks,  automatically 
send  and  receive  notifications 

Collaborative 

n/a 

Proprietary  (VIP 
Quality  Software) 
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Tool 

Description 

Collaborative  or 
FEDEP  Focus 

Sub-Category/ 
FEDEP  level 
supported 

Open  Source/ 
Proprietary 
(Vendor) 

Virtual  OS 

Open  source  web  operating  system  or  distributed  web  desktop  developed  by  Advanced 

Webhosting  Network  that  creates  a  shared  virtual  desktop  environment  on  a  remote  server  to 
promote  collaboration,  simplify  content  management  and  potentially  reduce  the  effort  associated 
with  network  installation 

Collaborative 

Project 

Collaboration,  web 
based  solutions 

Open  Source 

Virtual  Program  Office  (VPO) 

A  Virtual  Program  Office  application  is  based  on  the  IBM/Lotus  Domino  technology,  and  was 
designed  to  enable  geographically  dispersed  teams  to  work  collaboratively  via  the  Internet  in  a 
secure  environment.  They  are:  secure,  accessible  to  anyone  who  has  prior  permission  including 
contractors,  supports  membership  access  levels,  organized  to  support  distributed  work,  and  can 
support  very  large  documents. 

Collaborative 

Mil/Gov 

Unknown 

Visual  OMT 

Visual  OMTT  is  a  project-based  multiple-document  (MDI)  application  supporting  Simulation  Object 
Models,  Federation  Object  Models  and  Data  Dictionary  documents.  Object-model  elements  can  be 
copied  within  and  between  documents  by  drag  and  drop. 

FEDEP 

(Development) 

3,  4,5 

Unknown 

VR  Link 

With  MAK’s  VR-Link  networking  toolkit  you  can  network  simulators  and  virtual  reality  applications 
together,  using  the  HLA. 

FEDEP 

(Development) 

4 

Unknown 

Webasyst 

A  suite  of  web  based  software  applications  for  group  online  collaboration. 

Collaborative 

n/a 

Proprietary 

(WebAsyst) 

WebCOP  (joint) 

Framework  for  developing  Distributed  GIS  capabilities.  Web  application  built  using  Java  and  ESRI 
Map  Objects  Java  Edition  (MOJE).  Provides  GUI;  handles  map  requests;  gathers  maps  from 
Distributed  Services;  displays  an  aggregated  map  image;  ArcIMS,  WMS  and  WFS  are  all 
supported  by  Joing  WebCOP 

Collaborative  -  CFEC 

n/a 

Unknown 

Web  Ex 

Collaborate  and  communicate  securely  over  the  web.  Share  documents,  make  presentations, 
demonstrate  products  and  collaborate  capabilities. 

Collaborative 

n/a 

Proprietary 

(WebEx) 

WiredRed  Web 

Secure,  multipoint  routing  technology  power  an  e/pop®  product  line,  which  includes  multiparty 
video  conferencing,  web  conferencing  and  secure  IM  software. 

Collaborative  -  CM 

Mil/Gov 

Proprietary 

(WiredRed) 

Workspot 

A  web  service  that  gives  a  Linux  desktop  (outside  your  network),  which  can  be  shared  securely 
through  a  browser. 

Collaborative 

n/a 

Proprietary 

(Workspot) 

WorkZone 

Easy-to-use  online  collaborative  software  for  securely  sharing  work  with  colleagues,  clients  and 
team  members. 

Collaborative 

n/a 

Proprietary 

(Trichys) 

yaRTI 

yaRTI  is  an  HLA  RTI  implemented  in  Ada95,  using  the  Distributed  Systems  Annex  features. 

FEDEP 

(Development) 

5,6 

Unknown 

Zimbra 

Open  source  messaging  and  collaboration  application.  The  powerful  web  client  integrates  email, 
contacts,  shared  calendar,  VoIP,  and  online  document  authoring  into  a  browser-based  interface. 
Zimbra  also  offers  over-the-air  "push"  synchronization  to  mobile  devices,  a  complete  high  security 
package  including  built-in  anti-spam  and  anti-virus  scanning  and  features  Archiving  and  Discovery 
to  save  and  search  email. 

Collaborative 

Groupware,  classic 
client-server 
solutions/ web  based 
solutions 

Open  Source 

Zing  Technologies  AnyZing 
and  ZingThing 

Electronic  Learning  and  Meeting  System  software  for  team  rooms  and  networks. 

Collaborative 

Mil/Gov 

Proprietary  (Zing 
Technologies  Pty. 
Ltd) 
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MAN  SYSTEMS 


Annex  C:  Applied  Evaluation  Matrices 


Tool: 

Groove 

Is  a  Microsoft  Windows  based  peer-to-peer 
platform.  Groove,  a  subsidiary  of  Microsoft, 
is  a  collaboration  tool  allowing  ‘•users  the 
ability  to  work  collectively  on  a  project. 
Multip'e  users  can  log  on  together  or  work 
individually  within  a  shared  space  that 
contains  the  information  (e.g.,  documents, 
etc.)  that  they  are  working  on  Users  will  not 
have  access  to  a  shared  space  or  associated 
information  unless  they  have  been  invited  to. 
and  accept,  the  invitation  to  participate.  If  a 
user  is  uninvited,  new  keys  are  distributed  by 
Groove  to  all  remaining  participants.’’  The 
Department  of  Homeland  Security  uses 
Groove,  primarily  because  of  its  encryption 
capabi  ity,  and  because  the  audio  is  also 
encrypted  The  U  S.  Department  of  State 
‘used  Groove  to  coordinate  the 
establishment  of  the  U.S.  Embassy  in 
Baghdad.  Gallaherand  O'Rourke  (2004) 
reported  that  27  percent  of  their  Naval 
Postgraduate  School  survey  respondents 
reported  using  Groove.  Additionally,  the  U.S. 
Army  Corps  of  Engineers  uses  Groove  as 
part  of  its  common  operating  environment  tha 


Uses  / 
Goals 


Requirements 


Communication 

Design 
File  shanng 
File  storage 
Archiving 

Simulation/demonstration 

Testing/validation 

Planning 

Project  monitoring/control 
Workflow  control 
Security/access  control 
Social  activities  and  team  building 
Automated  project  support 
Resource  allocation 
Decision  support 
Analysis  support 
Team  coordination 
Software  coding/programming 
Network  development 

Total  Score: 
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H  U  M  A  N  S  )'S  MS 


Tool: 

Google  Docs 

If  you've  ever  struggled  to  keep  track  of 
different  versions  of  spreadsheet  or  word 
processor  files  sent  by  email.  Google  Docs  & 
Spreadsheets  may  be  right  for  you.  Google 
Docs  &  Spreadsheets  is  a  free  web-based 
word  processing  and  spreadsheet  program 
that  keeps  documents  current  and  lets  the 
people  you  choose  update  files  from  their 
own  computers.  You  can,  for  example, 
coordinate  your  student  group's  homework 
assignments,  access  your  family  to-do  list 
from  work  or  home,  or  collaborate  with 
remote  colleagues  on  a  new  business  plan. 
Import  your  existing  documents  and 
spreadsheets,  or  create  new  ones  from 
scratch. 

Edit  your  documents  and  spreadsheets  from 
anywhere.  All  you  need  is  a  Web  browser  - 
your  documents  and  spreadsheets  are 
stored  securely  online. 

Share  changes  in  real-time.  Invite  people  to 
your  documents/spreadsheets  and  make 
changes  together,  at  the  same  time. 


Uses/ 

Goals 


Communication 

Design 
File  sharing 
File  storage 
Archiving 

Simulation/demonstration 

Testing/validation 

Planning 

Project  monitoring/control 
Workflow  control 
Security/access  control 
Social  activities  and  team  building 
Automated  project  support 
Resource  allocation 
Decision  support 
Analysis  support 
Team  coordination 
Software  coding/programming 
Network  development 

Total  Score: 
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Requirements 


Tod: 

LiveLnk 

Livelink  ECM  -  Collaboration  combines  a 
robust  knowledge  repository  with  project 
workspaces,  polls,  news  channels,  tasks  and 
milestones  to  provide  all  of  the  tools  that  you 
need  to  collaborate  more  effectively. 
Spec.aiized  forums,  blogs,  newsletter  views, 
FAQs  and  events  calendars  help  promote 
shared  expertise  and  best  practices  within 
your  organization  and  beyond.  Collaborative 
tools  and  content  co-exist  in  an  intuitive. 
Web-based  environment  to  enhance  your 
team  and  community  collaboration 
experience. 

Document  Management  is  a  document 
management  solution  that  provides  full 
lifecycle  management  for  any  type  of 
electronic  document.  Document 
Management  provides  a  single,  authoritative 
repository  for  storing  and  organizing 
electronic  documents. 

Advanced  Workflow  combines  a  secure 
document  repository  and  information  retrieval 
with  state-of-the-art  process  automation 
tools.  Define,  apply,  and  reuse  business  logic 
to  improve  your  organizational  processes. 
http://www.opentext  coni 
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HUMANSrsrO/S 


Too  : 

[.rtfv'oo:  r? 

A  suite  of  collaborative  tools  that  leverage 
PCs  and  the  internet.  Including  video  and 
audio  conferencing,  chat,  an  internet 
directory,  file  transfer,  program  sharing, 
remote  desktop  sharing,  security,  advanced 
calling  (ability  to  send  mail  to  NetMeetng 
user  or  initiate  NetMeeting  call  directly  from 
address  book),  and  whiteboard  The 
application  also  supports  multipoint  data 
conferencing  and  internet  telephony/audio  & 
video  conferencing.  System  requirements 
are  minimal  (by  today's  standards)  but 
experience  has  shown  that  the  higher  the 
system  spec,  the  better. 
http://www.microsoft.  conVwindows/netmeetn 
9 1 


Uses  / 
Goals 


Communication 

Design 
File  sharing 
File  storage 
Archiving 

Simulation/demonstration 

Testing/validation 

Planning 

Project  monrtoring/control 
Workflow  control 
Security/access  control 
Social  activities  and  team  building 
Automated  project  support 
Resource  allocation 
Decision  support 
Analysis  support 
Team  coordination 
Software  coding/programming 
Network  development 

Total  Score: 
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Tool: 
share  :o  rt 

Microsoft  Office  SharePoint  Server  2007  is 
an  integrated  suite  of  server  capabilities  that 
can  help  improve  organizational 
effectiveness  by  providing  comprehensive 
content  management  and  enterpnse  search, 
acceierat  ng  shared  business  processes,  and 
facilitating  information-sharing  across 
boundaries  for  better  business  insight.  Office 
SharePoint  Server  2007  supports  all  intranet, 
extranet,  and  Web  applications  across  an 
enterprise  within  one  integrated  platform, 
of  relying  on  separate  fragmented 
Additionally,  this  collaboration  and 
content  management  server  provides  IT 
professions  s  and  developers  with  the 
platform  and  tools  they  need  for  server 
administration,  application  extensibility,  and 
interoperability. 

Manage  content  and  processes,  improve 
business  insight,  simplify  internal  and 
external  collaboration,  empower  IT  to  make  a 
strategic  impact 
http://office.mi  crosoft.com/en- 
us/sharepointserver/HAI  01 656531 033.aspx 


Uses/ 

Goals 


Requirements 


Communication 

Design 
File  shanng 
File  storage 
Archiving 

Simulation/demonstration 

Testing/validation 

Planning 

Project  monitoring/control 
Workflow  control 
Security/access  control 
Social  activities  and  team  building 
Automated  project  support 
Resource  allocation 
Decision  support 
Analysis  support 
Team  coordination 
Software  coding/programming 
Network  development 

Total  Score: 
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(U)  The  objectives  of  this  work  were  (1)  to  survey  the  marketplace  for  available  tools  that  may 
provide  collaborative  environments  to  support  Synthetic  Environment  based  exercises  and 
experiments,  and  evaluate  the  most  relevant  candidates  and  (2)  to  develop  an  evaluation 
method  for  assessing  collaborative  planning  and  engineering  tools.  A  literature  review  was 
conducted,  followed  by  Subject  Matter  Experts  (SMEs)  interviews.  A  total  of  215  collaborative 
tools  were  uncovered. 
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the  tools  known  prior  to  tool  selection.  Further,  specific  requirements  would  be  desired  of  the 
tools.  This  led  to  the  creation  of  the  Evaluation  Matrix  which  was  used  to  evaluate  select 
collaborative  tools.  Research  and  development  opportunities  can  also  be  identified  through  the 
evaluation  matrix.  Finally,  a  number  of  changes  are  proposed  for  the  evaluation  matrix.  The 
authors  feel  that  this  project  has  provided  an  important  first  step  toward  the  technological 
support  of  distributed  planning  and  engineering  teams. 
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